Robot learning holds tremendous promise to unlock the full potential of flexible, general, and dexterous robot systems, as well as to address some of the deepest questions in artificial intelligence. However, bringing robot learning to the level of generality required for effective real-world systems faces major obstacles in terms of data, generalization, and robustness. In this paper, we discuss how generalist robot policies (i.e., robot foundation models) can address these challenges, and how we can design effective generalist robot policies for complex and highly dexterous tasks. We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We then discuss how this model can be trained on a large and diverse dataset from multiple dexterous robot platforms, including single-arm robots, dual-arm robots, and mobile manipulators. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people and from a high-level VLM policy, and its ability to acquire new skills via fine-tuning. Our results cover a wide variety of tasks, such as laundry folding, table cleaning, and assembling boxes.
核心贡献 · Key contributions
提出π0，一种用于通用机器人控制的新型视觉-语言-动作流模型。 Proposes π0, a novel vision-language-action flow model for general robot control.
将预训练 VLM 与流匹配结合，实现高频灵巧动作块。 Integrates pre-trained VLM with flow matching for high-frequency dexterous action chunks.
在 7 种机器人平台上使用 10,000 小时数据进行跨本体训练。 Demonstrates cross-embodiment training on 7 robot platforms with 10,000 hours of data.
展示预训练后的零样本任务性能和语言指令跟随能力。 Shows zero-shot task performance and language instruction following after pre-training.
在叠衣服和组装盒子等复杂灵巧任务上取得最先进结果。 Achieves state-of-the-art results on complex dexterous tasks like laundry folding and box assembly.
引入类似大语言模型的预训练/后训练方案用于机器人基础模型。 Introduces a pre-training/post-training recipe analogous to LLMs for robot foundation models.
局限 · Limitations
对最优预训练数据组成和权重的理解有限。 Limited understanding of optimal pre-training data composition and weighting.
并非所有评估任务都达到可靠性能；仍存在失败模式。 Not all evaluated tasks achieve reliable performance; failure modes remain.
不清楚如何预测接近完美性能所需的数据量和类型。 Unclear how to predict data amount and type needed for near-perfect performance.
跨高度多样化领域（如驾驶、导航）的正向迁移尚未验证。 Positive transfer across highly diverse domains (e.g., driving, navigation) not yet validated.
模型大小和计算成本可能限制在资源受限机器人上的实时部署。 Model size and computational cost may limit real-time deployment on resource-constrained robots.