We introduce Devstral-Small, a lightweight open source model for code agents with the best performance among models below 100B size. In this technical report, we give an overview of how we design and develop a model and craft specializations in agentic software development. The resulting model, Devstral-Small is a small 24B model, fast and easy to serve. Despite its size, Devstral-Small still attains competitive performance compared to models more than an order of magnitude larger.
核心贡献 · Key contributions
提出 Devstral-Small，一个 24B 的开源模型，专为代码智能体任务设计，在 100B 以下模型中达到最优性能。 Introduces Devstral-Small, a 24B open-source model specialized for code agent tasks, achieving state-of-the-art among sub-100B models.
提出两阶段训练流程，结合智能体轨迹上的监督微调和策略优化。 Proposes a two-stage training pipeline combining supervised fine-tuning on agent trajectories and policy optimization.
开发了迭代评估协议，最多 3 次尝试并调度温度，以稳健测量智能体性能。 Develops an iterative evaluation protocol with up to 3 attempts and temperature scheduling for robust agent performance measurement.
证明专用代码智能体模型在软件工程任务上可超越更大的通用模型。 Demonstrates that specialized code agent models can outperform much larger general models on software engineering tasks.
发布改进版 Devstral-Small-2507，采用精炼的数据筛选和伪脚手架训练。 Releases an improved version Devstral-Small-2507 with refined data curation and pseudo-scaffold training.
表明 50 次迭代平衡效率与性能，较低温度从多次尝试中获益。 Shows that 50 iterations balance efficiency and performance, and lower temperatures benefit from multiple attempts.
局限 · Limitations
评估仅限于 SWE-bench Verified，未验证对其他基准或现实任务的泛化性。 Evaluation limited to SWE-bench Verified; generalization to other benchmarks or real-world tasks is not validated.
模型依赖特定脚手架（OpenHands），在不同智能体框架下性能可能变化。 Model relies on specific scaffold (OpenHands); performance may vary with different agent frameworks.
未使用检索工具或多样本投票，结果可能未反映最佳智能体性能。 No retrieval tool or multi-sample voting used; results may not reflect best-case agent performance.
训练数据限于 SWE-Gym 轨迹，未保证覆盖多样化的软件工程场景。 Training data limited to SWE-Gym trajectories; coverage of diverse software engineering scenarios is not guaranteed.
迭代评估协议可能未完全消除随机性，更高级的优化策略留待未来工作。 Iterative evaluation protocol may not fully eliminate stochasticity; more advanced refinement strategies are left for future work.
论文章节 · Sections（共 14）
摘要Abstract
1 引言1 Introduction
2.1 基础模型2.1 Base Model
2.2 数据2.2 Data
2.3 后训练2.3 Post-training
3 实验3 Experiments
3.1 设置3.1 Setup
3.2 结果3.2 Results
3.3 分析3.3 Analysis
3.4 温度缩放3.4 Temperature Scaling
4 刷新 Devstral-Small 数据4 Refreshing Devstral-Small Data