We introduce Nemotron-Nano-9B-v2, a hybrid Mamba-Transformer language model designed to increase throughput for reasoning workloads while achieving state-of-the-art accuracy compared to similarly-sized models. Nemotron-Nano-9B-v2 builds on the Nemotron-H architecture, in which the majority of the self-attention layers in the common Transformer architecture are replaced with Mamba-2 layers, to achieve improved inference speed when generating the long thinking traces needed for reasoning. We create Nemotron-Nano-9B-v2 by first pre-training a 12-billion-parameter model (Nemotron-Nano-12B-v2-Base) on 20 trillion tokens using an FP8 training recipe. After aligning Nemotron-Nano-12B-v2-Base, we employ the Minitron strategy to compress and distill the model with the goal of enabling inference on up to 128k tokens on a single NVIDIA A10G GPU (22GiB of memory, bfloat16 precision). Compared to existing similarly-sized models (e.g., Qwen3-8B), we show that Nemotron-Nano-9B-v2 achieves on-par or better accuracy on reasoning benchmarks while achieving up to 6x higher inference throughput in reasoning settings like 8k input and 16k output tokens. We are releasing Nemotron-Nano-9B-v2, Nemotron-Nano12B-v2-Base, and Nemotron-Nano-9B-v2-Base checkpoints along with the majority of our pre- and post-training datasets on Hugging Face.
核心贡献 · Key contributions
提出 Nemotron-Nano-9B-v2，一种混合 Mamba-Transformer 模型，在推理任务中吞吐量比同类模型高 6 倍。 Introduces Nemotron-Nano-9B-v2, a hybrid Mamba-Transformer model achieving up to 6x higher inference throughput than similarly-sized models for reasoning tasks.
使用 FP8 训练方案在 20 万亿 token 上预训练 12B 参数基础模型，实现高效的规模扩张。 Pre-trains a 12B-parameter base model on 20 trillion tokens using an FP8 training recipe, enabling efficient scaling.
采用 Minitron 剪枝与蒸馏将 12B 模型压缩至 9B，同时在单个 A10G GPU 上实现 128k 上下文推理。 Employs Minitron pruning and distillation to compress the 12B model to 9B while enabling 128k context inference on a single A10G GPU.
利用多阶段后训练，包括 SFT、GRPO、DPO 和 RLHF，并通过截断推理轨迹实现限时思考。 Utilizes multi-stage post-training including SFT, GRPO, DPO, and RLHF with truncated reasoning traces for budgeted thinking.
在 Hugging Face 上发布所有模型及大部分训练数据集，以促进可复现性。 Releases all models and most training datasets on Hugging Face for reproducibility.
在推理基准（如 MATH、AIME 和代码任务）上达到最先进准确率，同时保持高吞吐量。 Achieves state-of-the-art accuracy on reasoning benchmarks while maintaining high throughput, including MATH, AIME, and code tasks.
局限 · Limitations
通过剪枝进行的压缩可能会在蒸馏未优化的某些任务上降低准确率。 Compression via pruning may degrade accuracy on certain tasks not optimized for during distillation.
模型的效率提升在长思考推理任务上最为显著；在短上下文或非推理任务上的表现可能不那么突出。 The model's efficiency gains are most pronounced for long-thinking reasoning tasks; performance on short-context or non-reasoning tasks may be less impressive.
对特定 GPU 硬件（NVIDIA A10G）的依赖限制了向内存更少的其他设备的泛化能力。 Dependence on specific GPU hardware (NVIDIA A10G) limits generalizability to other devices with less memory.
模型训练严重依赖合成数据，可能引入自然数据中不存在的偏差或质量问题。 The model's training relies heavily on synthetic data, which may introduce biases or quality issues not present in natural data.
当 token 预算过紧时，限时思考机制可能导致不完整或不令人满意的回答。 The budgeted thinking mechanism may lead to incomplete or unsatisfactory responses when the token budget is too restricted.
论文章节 · Sections（共 28）
NVIDIA Nemotron Nano 2：一个准确且高效的混合 Mamba-Transformer 推理模型NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
1 引言1 Introduction
2 预训练2 Pretraining
2.1 模型架构2.1 Model Architecture
2.2 预训练数据2.2 Pre-Training Data
2.2.1 策展数据2.2.1 Curated Data
2.2.2 合成生成数据2.2.2 Synthetically-Generated Data
2.3 数据混合与排序2.3 Data Mixture and Ordering
2.3.1 多语言数据消融研究2.3.1 Multilingual Data Ablation Study
2.3.2 基础推理微调风格数据消融研究2.3.2 Fundamental Reasoning SFT-Style Data Ablation Study