Proof-Step Verification Raises Math Answer Precision but Sharply Cuts Coverage

A study of open-weight models tested a pipeline that rewrites solver answers into typed proof steps and has LLM judges certify or decline them. On GPQA and AIME-2025, certification lifted answer-key precision by 31 and 46 points, but the system answered only 35% and 21% of attempts. A self-consistency baseline matched much of the gain at similar coverage, and many failures came from invalid proof construction before any judge was reached.

Why it matters: Teams deploying LLM verifiers should weigh precision gains against lost coverage and report execution failures, selected-answer precision, and proof validity separately, since answer-key matches overstate proof correctness.