Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn general-purpose representations, and (2) large scale instruction tuning and reinforcement learning, to better align to end tasks and user preferences. We measure the relative importance of these two stages by training LIMA, a 65B parameter LLaMa language model fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses, without any reinforcement learning or human preference modeling. LIMA demonstrates remarkably strong performance, learning to follow specific response formats from only a handful of examples in the training data, including complex queries that range from planning trip itineraries to speculating about alternate history. Moreover, the model tends to generalize well to unseen tasks that did not appear in the training data. In a controlled human study, responses from LIMA are either equivalent or strictly preferred to GPT-4 in 43% of cases; this statistic is as high as 58% when compared to Bard and 65% versus DaVinci003, which was trained with human feedback. Taken together, these results strongly suggest that almost all knowledge in large language models is learned during pretraining, and only limited instruction tuning data is necessary to teach models to produce high quality output.
核心贡献 · Key contributions
表明仅用 1000 个精选示例微调 65B LLaMa 即可实现强对齐。 Shows that fine-tuning a 65B LLaMa on only 1,000 curated examples achieves strong alignment.
提出浅层对齐假说：知识来自预训练，对齐学习风格。 Proposes the Superficial Alignment Hypothesis: knowledge is from pretraining, alignment learns style.
LIMA 在 43% 的案例中匹配或优于 GPT-4，并超越经 RLHF 训练的 DaVinci003。 LIMA matches or beats GPT-4 in 43% of cases, and outperforms RLHF-trained DaVinci003.
表明不增加多样性的数据规模扩张收益递减。 Demonstrates that scaling data quantity without diversity yields diminishing returns.
表明仅添加 30 条对话链即可具备多轮对话能力。 Shows multi-turn dialogue ability with only 30 additional dialogue chains.
提供绝对评估：LIMA 88% 的响应满足提示要求，50% 为优秀。 Provides absolute evaluation: 88% of LIMA responses meet prompt requirements, 50% are excellent.
局限 · Limitations
手动策划 1000 个示例劳动密集且难以规模化。 Manual curation of 1,000 examples is labor-intensive and hard to scale.
LIMA 不如生产级模型鲁棒；对抗性提示可能导致弱响应。 LIMA is less robust than production models; adversarial prompts can cause weak responses.
限于单轮微调；多轮对话需要额外数据。 Limited to single-turn fine-tuning; multi-turn dialogue requires additional data.
安全性能有限；隐含恶意意图可能绕过防护。 Safety performance is limited; implicit malicious intents may bypass safeguards.
结果可能无法泛化到其他模型规模或架构，需进一步研究。 Results may not generalize to other model sizes or architectures without further study.
论文章节 · Sections（共 12）
摘要Abstract
1 引言1 Introduction
2 对齐数据2 Alignment Data
2.1 社区问答2.1 Community Questions & Answers
2.2 手动编写示例2.2 Manually Authored Examples
3 训练 LIMA3 Training LIMA
4 人工评估4 Human Evaluation
4.1 实验设置4.1 Experiment Setup
4.3 分析4.3 Analysis
5 为何少即是多？数据多样性、质量与数量的消融实验5 Why is Less More? Ablations on Data Diversity, Quality, and Quantity