AI Theorem Proving Papers · LLM Reasoning · AI for Science
AlphaGeometry vs DeepSeek-Prover vs MaxProof: Three Different Games
AlphaGeometry, DeepSeek-Prover-V1.5 and MaxProof all report headline math scores, but they prove different kinds of theorems under different verifiers and search budgets. Here is what each number actually measures.
The one thing to get straight first
“AI theorem prover” covers at least three systems that share almost no machinery. AlphaGeometry solves olympiad geometry by pairing a language model with a symbolic deduction engine, and the symbolic engine checks every step, so a proof is either valid or it is not. DeepSeek-Prover-V1.5 writes Lean 4 proofs and uses the proof assistant’s pass/fail signal as a reinforcement learning reward, then searches with a Monte Carlo tree variant at inference time. MaxProof tackles human-graded contest proofs where no fast symbolic checker exists, so it has to scale test time: many candidate proofs, a generative verifier, a repair loop, and a final ranker. The systems sit on a spectrum of how trustworthy the verifier is, and that single design choice explains most of the score differences below.
A fourth line of work, AI-aided formal proof search, applies the generate-and-verify loop to open research problems rather than contests. It is included because it exposes the cost and scaling behavior that the benchmark papers hide.
Key numbers
| System | What it proves | Benchmark | Score | Baseline in the same paper | Verification layer |
|---|---|---|---|---|---|
| AlphaGeometry | IMO-level Euclidean geometry, own formal language | 30 olympiad geometry problems, 2000 to 2022 | 25/30 | Human gold-medal average 25.9; prior best automated method 10; unguided symbolic engine 18 | Symbolic deduction engine checks every step |
| DeepSeek-Prover-V1.5 | Formal Lean 4 theorems, high-school level | miniF2F test | 63.5% | Improves over DeepSeek-Prover-V1 by changing both training and inference | Lean execution gives hard pass/fail |
| DeepSeek-Prover-V1.5 | Formal Lean 4 theorems, undergraduate level | ProofNet | 25.3% | Same system, harder benchmark | Lean execution |
| MaxProof around MiniMax M3 | Human-graded contest proofs | IMO 2025 | 35/42, above the gold-medal threshold | System-level result; not a single-sample score | Generative verifier plus repair loop, no symbolic checker |
| MaxProof around MiniMax M3 | Human-graded contest proofs | USAMO 2026 | 36/42, above the gold-medal threshold | Same search configuration | Generative verifier |
| AI formal proof search | Open research problems, Lean-style verification | Erdos open problems | 9/353 resolved | A basic LLM-plus-verifier loop reproduces the successes but is costlier on the hardest problems | Formal checker per attempt |
| AI formal proof search | OEIS conjectures | OEIS conjectures | 44/492 proved | Same | Formal checker |
Two rows deserve a second look. AlphaGeometry’s 25/30 is on a fixed 30-problem set curated from 2000 to 2022, and the paper’s own comparison points are in the same table: human gold medalists average 25.9, the previous best automated method solved 10, and the symbolic engine alone without the language model plateaus at 18. The 7-point jump over the symbolic baseline is the contribution, not the raw 25. MaxProof’s 35/42 is a search-system score from a typical configuration of 32 initial candidates, 4 verifier samples per candidate, 10 refinement rounds, and 4 new children per round. Comparing it with a pass@1 model number is a category error.
Why the same field reports 25/30, 63.5%, and 35/42
The scores are not on one leaderboard for a reason. Geometry with a clean formal language is decidable: the symbolic engine can confirm every deduction, which is why AlphaGeometry could be trained on 100 million synthetic theorems with machine-generated proofs and evaluated by a former IMO medalist who judged the outputs to be valid, human-readable proofs rather than coordinate bashing. Lean 4 is also machine-checked, but the search space is brutal: a proof can fail on library names, missing lemmas, or tactic syntax rather than mathematical ignorance, which is exactly the brittleness the DeepSeek-Prover page flags. Contest long-form proof is the hardest to verify of all, because correctness is an argument, not a term reduction. MaxProof’s entire design is an admission that its verifier is the weak link.
This is also why the benchmark layer matters. MiniF2F is a cross-system benchmark of 488 formalized Olympiad-level statements, built so that Lean, Isabelle, and HOL Light systems can be compared at all; the number 63.5% is only meaningful because a shared benchmark exists. LeanDojo contributes the interaction side: a Lean environment with premise retrieval, because a prover that cannot see the library it is working in fails before it starts.
The verifier is the bottleneck, and MaxProof shows why
The most instructive number in this comparison is not a score. In an earlier MiniMax cycle, the team ran a cross-verification cohort of 30 rollouts that a single-rubric verifier had marked perfect; an expert human judge marked only 17% fully correct, with 50% partially correct and 33% incorrect. A verifier with that false-positive rate, used as a reward signal, teaches the policy to produce long, well-formatted, judge-pleasing proofs that independent review rejects.
MaxProof’s response is pessimistic aggregation: bad-case filtering, solution normalization, multiple rubric and no-rubric judges, and a fitness score that errs toward rejection. The payoff shows up in the failure case the paper itself reports. On USAMO 2026 problem 2, the candidate archive contained a 6/7 oracle proof, but the ranker’s self-selected answer scored 2/7 because the tournament preferred a worse proof. On IMO 2025 the picture is cleaner: 7/7 on each of problems 1 through 5 and 0/7 on problem 6. The remaining error is selection under clustered verifier scores, not generation. Contrast the two regimes: DeepSeek-Prover gets its correctness signal free from Lean, and spends its engineering on RL and RMaxTS search; MaxProof has to spend its engineering on making an imperfect judge conservative enough to trust.
Cost, search, and the open-problem reality check
The formal proof search paper reports the cost side that contest benchmarks omit: a few hundred dollars per problem for the strongest system on Erdos problems. That is cheap for research triage and expensive for casual exploration. It also shows the loop generalizing modestly, 9 of 353 Erdos problems and 44 of 492 OEIS conjectures, and notes that a simple LLM-plus-verifier loop can replicate the headline successes, with the gap appearing on the hardest problems. Read together with AlphaGeometry’s 100 million synthetic theorems and MaxProof’s 32-candidate search populations, the pattern is consistent: verified math AI is a search business, and compute moves between training data, RL, and inference time rather than disappearing.
When to use which
- Your domain has a sound, decidable symbolic checker: neuro-symbolic search in the AlphaGeometry style. Synthesize verified training data, let a model propose creative leaps, and let the engine check every step. This is the only regime where the verifier is free and exact.
- Your work lives in a formal library like Lean or Isabelle: verifier-feedback RL plus tree search in the DeepSeek-Prover style, with retrieval as in LeanDojo. Budget for the search; a single greedy sample understates what these systems can do.
- Your output is human-graded long-form reasoning: population-level test-time scaling in the MaxProof style, and treat verifier calibration as the primary engineering risk. Assume single-rubric judges overrate, and measure the false-positive rate directly before trusting the reward.
- You are triaging open research questions: the generate-and-verify loop with a formal checker, at a few hundred dollars per problem. Reproduce the simple loop first; the expensive system earns its keep only on the hardest cases.
Limits and open questions
None of these numbers transfer across protocols. AlphaGeometry’s 25/30 is 30 geometry problems in its own formal language, not combinatorics or number theory, and problems must be pre-translated into its input. DeepSeek-Prover’s 63.5% miniF2F is high-school formal math in Lean 4, while its own 25.3% on undergraduate ProofNet shows how fast scores fall as statements get harder. MaxProof’s 35/42 is an IMO 2025 score from a specific 32-candidate search configuration and a self-reported grading pipeline, and the paper’s own USAMO problem 2 case shows the ranker can still pick a 2/7 proof over a 6/7 one. The open-problem numbers, 9/353 and 44/492, are modest on purpose: they count verified resolutions of statements nobody had proved before. The open question that cuts across all four systems is whether the recipes generalize to domains without geometry’s clean decidability or Lean’s industrial-strength kernel, because that, not another leaderboard point, is what decides whether this field is a collection of demos or a research tool.
FAQ
Which AI theorem prover is the best?
There is no single ranking because the systems solve different problems under different verifiers. AlphaGeometry scores 25/30 on olympiad geometry with a symbolic checker; DeepSeek-Prover-V1.5 scores 63.5% on the miniF2F formal benchmark in Lean 4; MaxProof scores 35/42 on IMO 2025 with human-style grading and heavy test-time search. Cross-comparing the raw numbers ignores the protocol differences this page catalogs.
Is AlphaGeometry better than human gold medalists?
Near, not above. On the same 30 olympiad geometry problems, AlphaGeometry solves 25 and the average human IMO gold medalist solves 25.9. It clearly beats the prior best automated method, which solved 10.
Why is DeepSeek-Prover-V1.5’s ProofNet score so much lower than its miniF2F score?
63.5% on miniF2F versus 25.3% on ProofNet reflects the benchmarks, not a bug. miniF2F targets high-school level formal statements while ProofNet targets undergraduate mathematics, so the gap is a difficulty signal, and the paper’s own limits section notes that Lean proofs can fail on library and syntax issues rather than mathematical content.
Can MaxProof’s IMO 2025 score be compared with a pass@1 model score?
No. MaxProof is a system-level result: a typical configuration generates 32 initial candidates, samples the verifier 4 times per candidate, and runs 10 refinement rounds with 4 new children per round. The paper itself warns it should not be compared casually with single-sample scores.
What is the biggest unsolved problem in AI theorem proving?
Verifier trust at scale. When a single-rubric judge marked 30 perfect-score rollouts, an expert judged only 17% fully correct. Any system that scales test-time search with a generative verifier inherits that false-positive risk, which is why the conservative-verifier designs in MaxProof and the free exact verifiers in Lean-style systems are the two most important lines of work.