We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4.
核心贡献 · Key contributions
开发了 GPT-4，一个接受图像和文本输入并生成文本输出的大规模多模态模型。 Developed GPT-4, a large multimodal model accepting image and text inputs, producing text outputs.
在专业和学术基准上达到人类水平表现，包括模拟律师考试前 10%。 Achieved human-level performance on professional and academic benchmarks, including top 10% on simulated bar exam.
构建了可预测的规模扩张基础设施，从较小模型准确预测 GPT-4 的损失和能力。 Built predictable scaling infrastructure, accurately forecasting GPT-4's loss and capabilities from smaller models.
通过基于人类反馈的强化学习（RLHF）和基于规则的奖励模型的后训练，提高了事实性和对齐。 Improved factuality and alignment via post-training with RLHF and rule-based reward models.
展示了强大的多语言性能，在翻译后的 MMLU 上 24 种语言超越英语最先进水平。 Demonstrated strong multilingual performance, surpassing English state-of-the-art on translated MMLU in 24 languages.
通过对抗性测试和模型辅助安全流程降低安全风险，将违禁内容响应减少 82%。 Reduced safety risks through adversarial testing and model-assisted safety pipeline, cutting disallowed content responses by 82%.
局限 · Limitations
不完全可靠；会幻觉事实并犯推理错误，在高风险场景中需谨慎。 Not fully reliable; hallucinates facts and makes reasoning errors, requiring caution in high-stakes contexts.
上下文窗口有限且不从经验中学习，与早期 GPT 模型类似。 Limited context window and does not learn from experience, similar to earlier GPT models.
缺乏 2021 年 9 月预训练数据截止后的事件知识。 Lacks knowledge of events after September 2021 pre-training data cutoff.
后训练降低了校准度；模型可能对错误预测过度自信。 Post-training reduces calibration; model can be overconfident in wrong predictions.
易受越狱和对抗性输入攻击，绕过安全缓解措施。 Vulnerable to jailbreaks and adversarial inputs that bypass safety mitigations.
论文章节 · Sections（共 12）
摘要Abstract
1 引言1 Introduction
2 本技术报告的范围与局限2 Scope and Limitations of this Technical Report
3 可预测的扩展3 Predictable Scaling
3.1 损失预测3.1 Loss Prediction
3.2 HumanEval 上的能力扩展3.2 Scaling of Capabilities on HumanEval
4 能力4 Capabilities
4.1 视觉输入4.1 Visual Inputs
5 局限性5 Limitations
6 风险与缓解措施6 Risks & mitigations
7 结论7 Conclusion
作者、署名与致谢Authorship, Credit Attribution, and Acknowledgements