Hy-Embodied-VLM-1.0：高效的物理世界智能体

Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

孙兴武 Xingwu Sun · · 2026-07-14 · arXiv:2607.12894 ↗ · 被引 2

摘要 · Abstract

构建有能力的具身智能体不仅需要多模态感知和理解，还需要对动作进行推理、适应不断变化的情况以及与物理世界互动的智能体能力。在本报告中，我们介绍了 Hy-Embodied-VLM-1.0，这是一个高效且强大的具身基础模型，专门为在物理世界中运行的具身智能体设计。为了从预训练阶段开始培养这些能力，我们定义了一个以动作为中心的能力分类法，包含三个渐进维度：与动作相关的状态理解、动作转换推理以及顺序和自适应推理。在该分类法的指导下，我们开发了系统化的数据流程，并策划了涵盖预训练和后训练的数据组合。为了提供强大的物理世界理解和交互能力，同时支持延迟敏感部署，我们在 Hy3-A3B 语言主干和 Hy-ViT2 视觉编码器上构建了模型。其高效的混合专家架构将强大的模型容量与高推理效率相结合。我们在涵盖具身感知、物理世界理解和具身推理的 38 个基准测试的综合套件上评估了 Hy-Embodied-VLM-1.0。该模型在 38 个基准测试中的 19 个上取得了同类尺寸模型中的最佳性能，并显著优于包括 Qwen3.6-A3B 和 Cosmos 3 在内的强大竞争对手。与上一代 Hy-Embodied-0.5 MoT-2B 相比，Hy-Embodied-VLM-1.0 的平均性能提升了 8.4%。尽管仅激活了 3B 参数，但其性能接近具有 32B 激活参数的上一代模型。除了静态基准评估，Hy-Embodied-VLM-1.0 在需要多轮交互和长程推理的具身智能体任务上也表现出强大的性能。

Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, adapting to evolving situations, and interacting with the physical world. In this report, we introduce Hy-Embodied-VLM-1.0, an efficient and powerful embodied foundation model specifically designed for embodied agents operating in the physical world. To cultivate such capabilities from the pre-training stage onward, we define an action-centric capability taxonomy comprising three progressive dimensions: Action-Relevant State Understanding, Action-Transition Reasoning, and Sequential and Adaptive Reasoning. Guided by this taxonomy, we develop a systematic data pipeline and curate data mixtures spanning both pre-training and post-training. To deliver strong physical-world understanding and interaction capabilities while supporting latency-sensitive deployment, we build our model on the Hy3-A3B language backbone and the Hy-ViT2 vision encoder. Its efficient Mixture-of-Experts architecture combines strong model capacity with high inference efficiency. We evaluate Hy-Embodied-VLM-1.0 on a comprehensive suite of 38 benchmarks covering embodied perception, physical-world understanding, and embodied reasoning. The model achieves the best performance among similarly sized models on 19 of the 38 benchmarks and substantially outperforms strong competitors, including Qwen3.6-A3B and Cosmos 3. Compared with the previous-generation Hy-Embodied-0.5 MoT-2B, Hy-Embodied-VLM-1.0 improves average performance by 8.4%. Despite activating only 3B parameters, it achieves performance close to that of the previous-generation model with 32B activated parameters. Beyond static benchmark evaluation, Hy-Embodied-VLM-1.0 also demonstrates strong performance on embodied agentic tasks requiring multi-turn interaction and long-horizon reasoning.

核心贡献 · Key contributions

提出一个面向具身智能体的动作中心能力分类法，涵盖三个递进层次。
Proposes an action-centric capability taxonomy covering three progressive levels for embodied agents.
构建系统性数据流水线，为预训练和后训练阶段精心设计数据混合。
Builds a systematic data pipeline with curated mixtures for pre-training and post-training stages.
仅用 3B 激活参数，在 38 个基准测试中 19 个达到同类模型最优。
Achieves state-of-the-art on 19 of 38 benchmarks among similarly sized models with only 3B activated parameters.
平均超越 Qwen3.6-A3B 和 Cosmos 3 等强对手 4.4%，较前代提升 8.4%。
Outperforms strong competitors like Qwen3.6-A3B and Cosmos 3 by 4.4% average, and prior version by 8.4%.
在 R2R-CE 和 Matterport3D 目标导航任务中展示强大的闭环导航性能。
Demonstrates strong closed-loop navigation performance on R2R-CE and Matterport3D object goal tasks.
结合强化学习与拒绝采样微调，激发具身智能体的推理能力。
Integrates reinforcement learning and rejection-sampling fine-tuning to elicit embodied agentic reasoning.

局限 · Limitations

评估依赖模拟基准，现实世界部署的差距尚未探索。
Evaluation relies on simulated benchmarks; real-world deployment gaps remain unexplored.
数据流水线依赖大量人工标注社交互动与失败案例。
Data pipeline requires extensive manual annotation for social interaction and failure cases.
专家混合模型可能存在专家利用不均衡，影响效率。
Mixture-of-Experts may suffer from uneven expert utilization, affecting efficiency.
在训练分布之外，深度因果推理与物理常识能力有限。
Limited capability for deep causal reasoning and physical commonsense beyond training distribution.
对未见物体、环境或长尾任务的泛化性尚未充分测试。
Generalization to unseen objects, environments, or long-tail tasks is not thoroughly tested.

论文章节 · Sections（共 26）

Hy-Embodied-VLM-1.0: Efficient Physical-World Agents Hy-Embodied-VLM-1.0: Efficient Physical-World Agents
1 引言 1 Introduction
2.1 分类体系 2.1 Taxonomy
2.2.1 预训练数据 2.2.1 Pre-training Data
2.2.2 监督微调数据 2.2.2 Supervised Fine-tuning Data
2.2.3 强化学习数据 2.2.3 Reinforcement Learning Data
3 模型架构 3 Model Architecture
4 训练 4 Training
4.1 预训练 4.1 Pre-Training
4.2 中训练与监督微调 4.2 Mid-Training & SFT
4.3 强化学习 4.3 Reinforcement Learning
4.3.1 基于 GRPO 的策略优化 4.3.1 Policy Optimization with GRPO
4.3.2 任务感知的具身奖励 4.3.2 Task-Aware Embodied Rewards
4.3.3 第一阶段：推理激发强化学习 4.3.3 Stage I: Reasoning-Elicitation RL
4.3.4 第二阶段：奖励专业化强化学习 4.3.4 Stage II: Reward-Specialized RL
4.4 拒绝采样微调 4.4 Rejection-Sampling Fine-Tuning
4.4.1 通过拒绝采样合成推理数据 4.4.1 Reasoning Data Synthesis via Rejection Sampling
4.4.2 推理内化 4.4.2 Reasoning Internalization
5 评估 5 Evaluation
5.1 动作相关状态理解 5.1 Action-Relevant State Understanding
5.2 动作-转变推理 5.2 Action-Transition Reasoning
5.3 顺序与自适应推理 5.3 Sequential and Adaptive Reasoning
5.4 视觉-语言导航 5.4 Vision-Language Navigation
5.4.1 指令跟随导航在 R2R-CE 上 5.4.1 Instruction-Following Navigation on R2R-CE
5.4.2 MP3D 上的零样本目标目标导航 5.4.2 Zero-Shot Object-Goal Navigation on MP3D
6 结论 6 Conclusion

阅读逐段中英对照全文 →