Mathematicians question OpenAI proof review and formalization standards on October 8
TechCrunch reports that 719 manuscripts included proofs without adequate peer review, 42 percent without formalization and discrepancies between natural language proofs and Lean code.
TechCrunch reported on October 8 that mathematicians were questioning whether OpenAI's recent mathematical results met formal research standards. Of 719 released manuscripts, only ten included model chain of thought and 42 percent of the proofs had not been formalized. Researchers from Cambridge and King's College London found at least two discrepancies between a natural language proof and its Lean code for a problem derived from the Navier Stokes equations, and advised against trusting automatically formalized proofs without peer review. Mathematician Terence Tao criticized users who sought only to make the model solve a target problem without understanding the output. Harvard professor Melanie Wood said meaningful verification begins after release. The advisory group hosted by Princeton's Institute for Advanced Study had recommended machine readable metadata linking natural language and formal artifacts, but the report said OpenAI did not provide it. OpenAI did not directly answer these specific criticisms.