Autonomous agents have made great strides in specialist domains like Atari games and Go. However, they typically learn tabula rasa in isolated environments with limited and manually conceived objectives, thus failing to generalize across a wide spectrum of tasks and capabilities. Inspired by how humans continually learn and adapt in the open world, we advocate a trinity of ingredients for building generalist agents: 1) an environment that supports a multitude of tasks and goals, 2) a large-scale database of multimodal knowledge, and 3) a flexible and scalable agent architecture. We introduce MineDojo, a new framework built on the popular Minecraft game that features a simulation suite with thousands of diverse open-ended tasks and an internet-scale knowledge base with Minecraft videos, tutorials, wiki pages, and forum discussions. Using MineDojo's data, we propose a novel agent learning algorithm that leverages large pre-trained video-language models as a learned reward function. Our agent is able to solve a variety of open-ended tasks specified in free-form language without any manually designed dense shaping reward. We open-source the simulation suite, knowledge bases, algorithm implementation, and pretrained models (https://minedojo.org) to promote research towards the goal of generally capable embodied agents.
核心贡献 · Key contributions
提出 MineDojo 框架，包含数千个 Minecraft 中的开放式任务。 Introduces MineDojo, a framework with thousands of open-ended tasks in Minecraft.
提供互联网规模的知识库，包括 73 万+ YouTube 视频、6 千+ Wiki 页面和 34 万+ Reddit 帖子。 Provides an internet-scale knowledge base of 730K+ YouTube videos, 6K+ Wiki pages, and 340K+ Reddit posts.
提出 MineCLIP，一种在 YouTube 数据上训练的对比视频-语言模型，用于奖励学习。 Proposes MineCLIP, a contrastive video-language model trained on YouTube data for reward learning.
证明基于 MineCLIP 的智能体在 12 个任务上达到与手工设计奖励相当的性能。 Demonstrates that MineCLIP-based agents achieve competitive performance to hand-engineered rewards on 12 tasks.
引入一种使用学习到的视频-语言模型评估创造性任务的新协议。 Introduces a novel evaluation protocol for creative tasks using a learned video-language model.
开源模拟套件、知识库、算法和预训练模型以促进研究。 Open-sources simulation suite, knowledge bases, algorithms, and pretrained models to promote research.
局限 · Limitations
智能体学习仅在 12 个任务上评估，限制了泛化性。 Agent learning is evaluated on only 12 tasks, limiting generalizability.
MineCLIP 奖励模型可能无法泛化到 Minecraft 领域之外的任务。 MineCLIP reward model may not generalize to tasks outside Minecraft domain.
创造性任务评估依赖学习到的指标，可能不完全与人类判断一致。 Creative task evaluation relies on a learned metric, which may not perfectly align with human judgment.
该框架特定于 Minecraft，限制了在其他具身环境中的适用性。 The framework is specific to Minecraft, limiting applicability to other embodied environments.
MineCLIP 的计算开销需要优化技术以实现实际的强化学习训练。 Computational overhead of MineCLIP requires optimization techniques for practical RL training.
论文章节 · Sections（共 14）
摘要Abstract
1 引言1 Introduction
2 MineDojo 模拟器与基准套件2 MineDojo Simulator & Benchmark Suite
2.1 任务套件 I：程序化任务2.1 Task Suite I: Programmatic Tasks
2.2 任务套件 II：创意任务2.2 Task Suite II: Creative Tasks
2.3 入门任务集合2.3 Collection of Starter Tasks
3 互联网规模知识库3 Internet-scale Knowledge Base
4 大规模预训练的智能体学习4 Agent Learning with Large-scale Pre-training
4.1 在大规模视频上预训练 MineCLIP4.1 Pre-Training MineCLIP on Large-scale Videos
4.2 基于 MineCLIP 奖励的强化学习4.2 RL with MineCLIP Reward