In order for robots to be useful, they must perform practically relevant tasks in the real world, outside of the lab. While vision-language-action (VLA) models have demonstrated impressive results for end-to-end robot control, it remains an open question how far such models can generalize in the wild. We describe $π_{0.5}$, a new model based on $π_{0}$ that uses co-training on heterogeneous tasks to enable broad generalization. $π_{0.5}$\ uses data from multiple robots, high-level semantic prediction, web data, and other sources to enable broadly generalizable real-world robotic manipulation. Our system uses a combination of co-training and hybrid multi-modal examples that combine image observations, language commands, object detections, semantic subtask prediction, and low-level actions. Our experiments show that this kind of knowledge transfer is essential for effective generalization, and we demonstrate for the first time that an end-to-end learning-enabled robotic system can perform long-horizon and dexterous manipulation skills, such as cleaning a kitchen or bedroom, in entirely new homes.
核心贡献 · Key contributions
提出π0.5，一种通过异构任务协同训练实现开放世界泛化的视觉-语言-动作模型。 Proposes π0.5, a VLA model co-trained on heterogeneous tasks for open-world generalization.
首次展示端到端机器人系统在未见过的家庭中执行长时灵巧操作任务。 Demonstrates first end-to-end robotic system performing long-horizon dexterous tasks in unseen homes.
表明与多机器人数据、网络数据和高层语义预测的协同训练可实现广泛泛化。 Shows co-training with multi-robot data, web data, and high-level semantic prediction enables broad generalization.
引入分层推理：模型先预测高层子任务，再通过流匹配预测底层动作。 Introduces hierarchical inference: model predicts high-level subtask then low-level actions via flow matching.
验证跨本体迁移和网络数据对泛化到新物体和场景至关重要。 Validates that cross-embodiment transfer and web data are critical for generalization to new objects and scenes.
达到与在测试家庭上训练的模型相当的性能，通过协同训练弥合泛化差距。 Achieves comparable performance to model trained on test homes, bridging generalization gap via co-training.
局限 · Limitations
模型在陌生环境中仍会犯错，如难以打开的抽屉或橱柜。 Model still makes mistakes in unfamiliar environments like hard-to-open drawers or cabinets.
部分可观测性挑战，例如机器人手臂在擦拭任务中遮挡污渍。 Partial observability challenges, e.g., robot arm occluding spills during wiping tasks.
高层子任务推理易分心，例如反复开关抽屉。 High-level subtask inference can be distracted, e.g., repeatedly opening and closing a drawer.
上下文窗口和记忆有限；无法处理需要跨房间导航或物体记忆的任务。 Limited context window and memory; cannot handle tasks requiring navigation between rooms or object memory.
提示复杂度受限于训练数据；更复杂的指令需要更丰富的标注。 Prompt complexity limited by training data; more intricate instructions require richer annotations.
论文章节 · Sections（共 18）
摘要Abstract
一、引言I Introduction
二、相关工作II Related Work
三、预备知识III Preliminaries
四、π0.5 模型与训练方案IV The π0.5\pi_{0.5} Model and Training Recipe
四-A π0.5 架构IV-A The π0.5\pi_{0.5} architecture
四-B 离散与连续动作表示的融合IV-B Combining discrete & continuous action representations
四-C 预训练IV-C Pre-training
四-D 后训练IV-D Post-training
四-E 机器人系统细节IV-E Robot system details
五、实验评估V Experimental Evaluation
五-A π0.5 能否泛化到真实家庭环境？V-A Can π0.5\pi_{0.5} generalize to real homes?
五-B 泛化能力如何随场景数量扩展？V-B How does generalization scale with the number of scenes?
五-C 协同训练方案各部分的重要性？V-C How important is each part of our co-training recipe?
五-D π0.5 与其他 VLA 的比较？V-D How does π0.5\pi_{0.5} compare to other VLAs?
五-E 高层推理的重要性？V-E How important is high-level inference?