State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning. To diagnose the failures of current models and support research, we introduce GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math word problems. We find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution. To increase performance, we propose training verifiers to judge the correctness of model completions. At test time, we generate many candidate solutions and select the one ranked highest by the verifier. We demonstrate that verification significantly improves performance on GSM8K, and we provide strong empirical evidence that verification scales more effectively with increased data than a finetuning baseline.
核心贡献 · Key contributions
引入 GSM8K 数据集，包含 8.5K 个高质量、语言多样的小学数学应用题。 Introduces GSM8K, a dataset of 8.5K high-quality grade school math word problems with high linguistic diversity.
提出训练验证器判断模型生成解的正确性，提升多步推理能力。 Proposes training verifiers to judge correctness of model-generated solutions, improving multi-step reasoning.
证明验证方法随数据增加比微调基线扩展更有效。 Demonstrates verification scales more effectively with increased data than finetuning baseline.
显示验证在 GSM8K 上提供相当于 30 倍模型规模增长的性能提升。 Shows verification provides performance boost equivalent to 30x model size increase on GSM8K.
发现词级验证器优于解级验证器，并从语言建模辅助目标中受益。 Finds token-level verifiers outperform solution-level verifiers and benefit from language modeling auxiliary objective.
证明 dropout 正则化显著提升微调和验证性能。 Demonstrates dropout regularization significantly improves both finetuning and verification performance.
局限 · Limitations
生成过多候选解时，验证性能因对抗样本而下降。 Verification performance degrades when generating too many candidate solutions due to adversarial examples.
验证需要足够大的数据集才能优于微调基线。 Verification requires a sufficiently large dataset to outperform finetuning baseline.
研究仅限于小学数学问题，未验证更复杂推理的泛化性。 Study limited to grade school math problems; generalization to more complex reasoning not verified.
验证器可能依赖粗略启发式而非彻底推理，尤其当比生成器小时。 Verifier may rely on coarse heuristics rather than thorough reasoning, especially when smaller than generator.
生成器过度自信降低解多样性，多训练轮次后限制验证器效果。 Generator overconfidence reduces solution diversity, limiting verifier effectiveness after many training epochs.