Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence. Although current reinforcement learning algorithms can be readily applied to tasks similar to what they have been developed for, configuring them for new application domains requires significant human expertise and experimentation. We present DreamerV3, a general algorithm that outperforms specialized methods across over 150 diverse tasks, with a single configuration. Dreamer learns a model of the environment and improves its behavior by imagining future scenarios. Robustness techniques based on normalization, balancing, and transformations enable stable learning across domains. Applied out of the box, Dreamer is the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula. This achievement has been posed as a significant challenge in artificial intelligence that requires exploring farsighted strategies from pixels and sparse rewards in an open world. Our work allows solving challenging control problems without extensive experimentation, making reinforcement learning broadly applicable.
核心贡献 · Key contributions
提出 DreamerV3，一种使用固定超参数掌握多种领域的通用算法。 Proposes DreamerV3, a general algorithm mastering diverse domains with fixed hyperparameters.
展示了有利的缩放特性，更大的模型可提升性能和样本效率。 Demonstrates favorable scaling properties where larger models improve performance and data-efficiency.
在 7 个基准测试的 150 多个任务上超越专用算法。 Outperforms specialized algorithms across over 150 tasks in 7 benchmarks.
首个无需人类数据或课程即可在 Minecraft 中从头收集钻石的算法。 First algorithm to collect diamonds in Minecraft from scratch without human data or curricula.
引入 symlog 预测和鲁棒归一化技术，实现跨领域的稳定学习。 Introduces symlog predictions and robust normalization for stable learning across domains.
在连续控制、BSuite 和 Crafter 基准测试上达到最先进水平。 Achieves state-of-the-art on continuous control, BSuite, and Crafter benchmarks.
局限 · Limitations
DreamerV3 仅在 1 亿步内偶尔学会在 Minecraft 中收集钻石。 DreamerV3 only learns to sometimes collect diamonds in Minecraft within 100M steps.
为学习 Minecraft 提高了方块破坏速度，可能无法泛化。 Increases block breaking speed for Minecraft learning, which may not generalize.
缩放特性需要在更大模型规模下进一步验证。 Scaling properties need further validation at larger model sizes.
每个任务训练独立智能体，缺乏跨任务迁移学习。 Trains separate agents per task, lacking transfer learning across tasks.