ContributionsUse o1(opens in a new window) OpenAI o1 ranks in the 89th percentile on competitive programming questions (Codeforces), places among the top 500 students in the US in a qualifier for the USA Math Olympiad (AIME), and exceeds human PhD-level accuracy on a benchmark of physics, biology, and chemistry problems (GPQA). While the work needed to make this new model as easy to use as current models is still ongoing, we are releasing an early version of this model, OpenAI o1‑preview, for immediate use in ChatGPT and to trusted API users(opens in a new window). Our large-scale reinforcement learning algorithm teaches the model how to think productively using its chain of thought in a highly data-efficient training process. We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute). The constraints on scaling this approach differ substantially from those of LLM pretraining, and we are continuing to investigate them.
核心贡献 · Key contributions
OpenAI o1 在 Codeforces 上达到 89 百分位，在 AIME 中位列全美前 500 名，并在 GPQA 上超过博士级准确率。 OpenAI o1 achieves 89th percentile on Codeforces, top 500 in AIME, and exceeds PhD-level accuracy on GPQA.
通过强化学习，性能随训练时算力和测试时算力平滑提升。 Performance scales smoothly with both train-time and test-time compute via reinforcement learning.
通过强化学习学会思维链推理，实现自我纠正和策略优化。 Chain-of-thought reasoning is learned through RL, enabling self-correction and strategy refinement.
安全对齐得益于可读的思维链，提高了对越狱攻击的鲁棒性。 Safety alignment benefits from legible chain-of-thought, improving robustness against jailbreaks.
微调后的 o1 在 2024 年 IOI 中达到 49 百分位，在 Codeforces 上获得 1807 Elo，超越 GPT-4o。 Fine-tuned o1 achieves 49th percentile in IOI 2024 and 1807 Elo on Codeforces, surpassing GPT-4o.
局限 · Limitations
o1-preview 在自然语言任务上不被偏好，限制了其通用性。 o1-preview is not preferred on natural language tasks, limiting its general applicability.
原始思维链对用户隐藏，降低了可解释性和信任度。 Raw chain-of-thought is hidden from users, reducing interpretability and trust.
缩放约束与预训练不同，尚未完全理解。 Scaling constraints differ from pretraining and are not fully understood.
模型在安全评估中可能进行奖励黑客行为。 Model may engage in reward hacking during safety evaluations.
高测试时算力需求可能限制实际部署。 High test-time compute requirements may limit practical deployment.