Octo: An Open-Source Generalist Robot Policy
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→在多样机器人数据集上预训练的大规模策略有潜力改变机器人学习：无需从头训练新策略，这种通才机器人策略只需少量领域内数据即可微调，却能广泛泛化。然而，为了广泛适用于各种机器人学习场景、环境和任务，这类策略需要处理多样的传感器和动作空间，适应各种常用机器人平台，并能快速高效地微调到新领域。在这项工作中，我们旨在为开发开源、广泛适用的机器人操作通才策略奠定基础。作为第一步，我们引入了 Octo，一种基于 Transformer 的大型策略，在 Open X-Embodiment 数据集（迄今为止最大的机器人操作数据集）的 80 万条轨迹上训练。它可以通过语言命令或目标图像进行指令，并能在标准消费级 GPU 上数小时内有效微调到具有新感官输入和动作空间的机器人设置。在 9 个机器人平台的实验中，我们证明了 Octo 作为一种多用途策略初始化，可以有效地微调到新的观察和动作空间。我们还对 Octo 模型的设计决策（从架构到训练数据）进行了详细消融，以指导未来构建通才机器人模型的研究。
Large policies pretrained on diverse robot datasets have the potential to transform robotic learning: instead of training new policies from scratch, such generalist robot policies may be finetuned with only a little in-domain data, yet generalize broadly. However, to be widely applicable across a range of robotic learning scenarios, environments, and tasks, such policies need to handle diverse sensors and action spaces, accommodate a variety of commonly used robotic platforms, and finetune readily and efficiently to new domains. In this work, we aim to lay the groundwork for developing open-source, widely applicable, generalist policies for robotic manipulation. As a first step, we introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset, the largest robot manipulation dataset to date. It can be instructed via language commands or goal images and can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs. In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models.
在多样化机器人数据集上预训练的大型策略有潜力变革机器人学习：这类通用机器人策略无需从头训练新策略，只需少量领域内数据即可微调，却能广泛泛化。然而，为了在广泛的机器人学习场景、环境和任务中普遍适用，这些策略需要处理多样的传感器和动作空间，适应各种常用机器人平台，并能快速高效地微调到新领域。在这项工作中，我们旨在为开发开源、广泛适用的机器人操作通用策略奠定基础。作为第一步，我们引入了 Octo，一个基于 Transformer 的大型策略，在 Open X-Embodiment 数据集（迄今为止最大的机器人操作数据集）的 80 万条轨迹上训练。它可以通过语言命令或目标图像进行指令，并且可以在标准消费级 GPU 上几小时内有效微调到具有新传感器输入和动作空间的机器人设置。在 9 个机器人平台的实验中，我们证明了 Octo 作为一种多功能的策略初始化，可以有效地微调到新的观察和动作空间。我们还对 Octo 模型的设计决策进行了详细的消融研究，从架构到训练数据，以指导未来构建通用机器人模型的研究。
Large policies pretrained on diverse robot datasets have the potential to transform robotic learning: instead of training new policies from scratch, such generalist robot policies may be finetuned with only a little in-domain data, yet generalize broadly. However, to be widely applicable across a range of robotic learning scenarios, environments, and tasks, such policies need to handle diverse sensors and action spaces, accommodate a variety of commonly used robotic platforms, and finetune readily and efficiently to new domains. In this work, we aim to lay the groundwork for developing open-source, widely applicable, generalist policies for robotic manipulation. As a first step, we introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset, the largest robot manipulation dataset to date. It can be instructed via language commands or goal images and can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs. In experiments across 9 robotic platforms, we demonstrate that Octo serves as a versatile policy initialization that can be effectively finetuned to new observation and action spaces. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models.
机器人学习的常见方法是在针对特定机器人和任务收集的数据集上训练策略。这种从零开始的学习方式需要为每个任务投入大量的数据收集工作，且所得策略通常只能实现狭窄的泛化。原则上，从其他机器人和任务中收集的经验提供了一种可能的解决方案，使模型接触到多样化的机器人控制问题，从而可能提高下游任务的泛化能力和性能。然而，即使通用模型在自然语言处理和计算机视觉领域已变得无处不在，构建能够控制多种机器人、完成多种任务的类似“通用机器人模型”仍被证明具有挑战性。在机器人领域训练统一控制策略面临独特挑战，需要处理不同的机器人形态、传感器设置、动作空间、任务规格、环境和算力预算。
The common approach for robotic learning is to train policies on datasets collected for the specific robot and task at hand. Learning from scratch in this way requires significant data collection effort for each task, and the resulting policies usually exhibit only narrow generalization. In principle, collected experience from other robots and tasks offers a possible solution, exposing models to a diverse set of robotic control problems that may improve generalization and performance on downstream tasks. However, even as general-purpose models become ubiquitous in natural language and computer vision, it has proven challenging to build the analogous “general-purpose robot model” that can control many robots for many tasks. Training a unified control policy in robotics presents unique challenges, requiring handling different robot embodiments, sensor setups, action spaces, task specifications, environments, and compute budgets.
朝着这一方向，已有若干工作提出了机器人基础模型，这些模型直接将机器人观测映射到动作，并对新领域和新机器人提供零样本或少样本泛化。我们广泛地将这些模型称为“通用机器人策略”（GRPs），强调它们在跨任务、跨环境和跨机器人系统执行低级视觉运动控制的能力。例如，GNM 模型可泛化于不同的机器人导航场景，RoboCat 模型可处理不同机器人形态的目标条件任务，RT-X 模型可在五种机器人形态上执行语言条件操作。尽管这些模型代表了迈向真正“通用机器人模型”的重要步骤，但它们在多个重要方面仍受到限制：它们通常将下游用户限制在预定义且往往受限的输入观测集（例如单一摄像头流）；它们缺乏对新领域进行有效微调的支持；重要的是，这些模型中最大的模型并未向公众开放。
Towards this direction, several works have proposed robotic foundation models that directly map robot observations to actions and provide zero-shot or few-shot generalization to new domains and robots. We broadly refer to these models as “generalist robot policies” (GRPs), emphasizing their ability to perform low-level visuomotor control across tasks, environments, and robotic systems. For example, the GNM model generalizes across different robotic navigation scenarios, the RoboCat model handles different robot embodiments for goal-conditioned tasks, and the RT-X model performs language-conditioned manipulation across five robot embodiments. Although these models represent significant steps toward a true “general-purpose robot model,” they have been limited in multiple important aspects: they typically constrain downstream users to a pre-defined and often restrictive set of input observations, e.g., a single camera stream; they lack support for effective finetuning to new domains; and importantly, the largest of these models are not available to the general public.
我们设计了一个用于预训练通用机器人策略的系统，更适合下游机器人应用中接口的多样性。我们模型的核心是一个 Transformer 架构，它将任意输入 token（由观测和任务创建）映射到输出 token（随后解码为动作），并可在多样化的机器人和任务数据集上进行训练。无需额外训练，该策略即可接受不同的摄像头配置（例如工作台摄像头或腕部摄像头），控制不同的机器人，并可通过语言命令或目标图像进行引导——所有这些只需改变输入模型的 token 即可。最重要的是，该模型可通过添加适当的适配器并使用小型目标域数据集和可负担的算力预算进行微调，从而适应具有新感官输入、动作空间或形态的新机器人设置。
We design a system for pretraining generalist robot policies more suitable for the diversity of interfaces in downstream robotic applications. The core of our model is a transformer architecture that maps arbitrary input tokens (created from observations and tasks) to output tokens (then decoded into actions), which can be trained on a diverse dataset of robots and tasks. With no additional training, this policy can accept different camera configurations (e.g., workspace or wrist cameras), can control different robots, and can be guided via either language commands or goal images — all by simply changing which tokens are fed into the model. Most importantly, the model can be adapted to new robot setups with new sensory inputs, action spaces, or morphologies by adding appropriate adapters and finetuning with a small target domain dataset and an accessible compute budget.
我们的主要贡献是 Octo，一个基于 Transformer 的策略，在迄今为止最大的机器人操作数据集上进行了预训练：来自 Open X-Embodiment 数据集的 80 万条机器人演示。Octo 是第一个能够有效微调以适应新观测和动作空间的 GRP，也是第一个完全开源的通用机器人操作策略，包括训练流程、模型检查点和数据。最后，虽然构成 Octo 的各个组件——Transformer 骨干、对语言和目标图像规范的支持，以及用于建模表达性动作分布的扩散头——已在先前工作中讨论过，但将这些组件组合成一个强大的通用机器人策略是独特且新颖的。
Our primary contribution is Octo, a transformer-based policy pretrained on the largest robot manipulation dataset to date: 800k robot demonstrations from the Open X-Embodiment dataset. Octo is the first GRP that can be effectively finetuned to new observations and action spaces and the first generalist robot manipulation policy that is fully open-source, including the training pipeline, model checkpoints, and data. Finally, while the individual components that comprise Octo — a transformer backbone, support for both language and goal image specification, and a diffusion head to model expressive action distributions — have been discussed in prior work, the particular combination of these components into a powerful generalist robot policy is unique and novel.
我们通过在 4 个机构的 9 个机器人上进行的广泛实验证明，我们的组合系统在单臂和双臂操作任务的即插即用多机器人控制方面达到了最先进的性能，并且 Octo 可作为有效初始化，用于微调到具有新观测和动作空间的未见设置。在此过程中，我们仔细研究了预训练 GRP 时不同设计决策的影响；我们评估了数据分布、模型架构和策略公式的选择如何影响预训练 GRP 的质量。我们的评估突出了规模和灵活性的效用：我们最好的模型是在最广泛的数据混合上训练的，具有最少限制的归纳偏置，并且策略目标能够适应预训练数据中行为的多样性。
We demonstrate through extensive experiments on 9 robots across 4 institutions that our combined system leads to state-of-the-art performance for out-of-the-box multi-robot control for single and dual-arm manipulation tasks and that Octo can be used as an effective initialization for finetuning to unseen setups with new observation and action spaces. In the process, we carefully study the effect of different design decisions when pretraining GRPs; we evaluate how the choice of data distribution, model architecture, and policy formulation affects the quality of the pretrained GRP. Our evaluation highlights the utility of scale and flexibility: our best models are those trained on the widest data mixtures, with the least restrictive inductive biases, and with policy objectives that can fit the diversity of behaviors in the pretraining data.
在本文中，我们随论文发布了训练、使用、复现和微调 Octo 模型所需的全部资源。我们提供了具有 27M 和 93M 参数的预训练 Octo 模型检查点，这些模型开箱即用，支持多路 RGB 摄像头输入以及语言和目标图像任务规范。我们还提供了在新领域微调这些模型的脚本，以及完整的预训练流程，包括优化的数据加载器、用于多模态输入的 Transformer 实现，以及监控训练进度的工具。
Along with this paper, we release all resources required to train, use, reproduce, and fine-tune an Octo model. We provide pretrained Octo model checkpoints with 27M and 93M parameters that, out of the box, support multiple RGB camera inputs as well as both language and goal image task specification. We also provide scripts for fine-tuning these models on new domains, as well as our complete pretraining pipeline, including optimized data loaders, transformer implementations for multimodal inputs, and tools to monitor training progress.
许多工作利用从机器人收集的大规模轨迹数据集来训练策略，从早期使用自主数据收集来扩展策略训练的努力，到最近探索将基于 Transformer 的现代策略与大型演示数据集相结合的工作。这些工作主要关注单一具身，而 Octo 则在跨多个具身组装的机器人数据集上训练策略，从而增加了训练数据的有效规模，并允许对一系列机器人设置进行微调。
Many works train policies using a large dataset of trajectories collected from a robot, from early efforts using autonomous data collection for scaling policy training to more recent efforts that explore the combination of modern transformer-based policies with large demonstration datasets. These works primarily focus on a single embodiment, while Octo trains policies on robot datasets assembled across multiple embodiments, increasing the effective size of the training dataset and allowing finetuning to a range of robot setups.
最近，论文聚焦于拓宽机器人策略的泛化能力。多项工作利用多样化的非机器人数据或预训练的视觉-语言基础模型来提升策略对新场景和新任务的泛化能力。与 Octo 更相关的是最近在多个机器人具身数据上训练机器人策略的工作：GNM 模型在机器人导航设置上泛化，而 RoboCat 和 RT-X 控制多个单臂操作机器人。虽然这些模型取得了令人印象深刻的策略学习结果，但一个关键问题是它们缺乏灵活性：它们通常要求用户坚持使用预训练期间使用的感官输入和动作空间，并且不支持适应新的观察和动作空间。此外，最大的模型不公开可用。Octo 在多个方面与这些工作不同：它在更大、更多样化的机器人数据混合上训练，通过高效微调支持更广泛的下游应用，并且完全开源和可复现。
More recently, papers have focused on broadening the generalization abilities of robot policies. Multiple works leverage diverse non-robot data or pretrained vision-language foundation models to boost policy generalization to new scenes and tasks. More closely related to Octo are recent works that train robot policies across data from multiple robot embodiments: the GNM model generalizes across robot navigation setups while RoboCat and RT-X control multiple single-arm manipulation robots. While these models deliver impressive policy learning results, a key issue is their lack of flexibility: they typically require users to stick to the sensory inputs and action space used during pretraining and do not support adaptation to new observation and action spaces. Furthermore, the largest models are not publicly accessible. Octo differs from these works in multiple aspects: it is trained on a larger and more diverse robot data mix, it supports a wider range of downstream applications via efficient finetuning to new robot setups, and it is fully open source and reproducible.
Octo 的设计受到机器人模仿学习和可扩展 Transformer 训练方面几项近期进展的启发，包括使用去噪扩散目标进行动作解码，预测“动作块”（即未来动作序列），以及受可扩展视觉 Transformer 训练文献启发的模型布局和学习率调度。我们的工作首次在跨具身通用策略学习的背景下利用这些方法，并发现它们可以带来显著的性能提升。在我们的评估中，我们进行了消融实验以评估这些组件的重要性，并在附录 E 中提供了我们认为（不）重要的更全面列表；我们希望我们的发现对通用策略学习的未来研究有用。
Octo’s design is inspired by several recent advances in robot imitation learning and scalable transformer training, including the use of denoising diffusion objectives for action decoding, the prediction of “action chunks”, i.e., sequences of future actions, and model layouts and learning rate schedules inspired by the literature on scalable vision transformer training. Our work is the first to leverage these approaches in the context of learning cross-embodied generalist policies and we find that they can lead to substantial performance improvements. In our evaluation, we present ablations to assess the importance of these components, alongside a more comprehensive list of what we found to be (un)important in Appendix E; we hope our findings are useful for future research on generalist policy learning.
训练通用机器人策略的一个关键要素是机器人训练数据。与可以从网络抓取的视觉和语言数据不同，大规模获取机器人数据具有挑战性，通常涉及大量的硬件和人力投资。有多个大型机器人导航和自动驾驶数据集。近年来，也有多项努力构建规模和多样性不断增加的机器人操作数据集，这些数据集要么通过脚本化和自主策略收集，要么通过人类远程操作收集。Octo 在 Open X-Embodiment 数据集上训练，该数据集是最近的一项努力，汇集了许多上述机器人数据集。Open-X 数据集包含约 150 万个机器人片段，我们从中筛选出 80 万个用于 Octo 训练。我们注意到 RT-X 模型使用了更受限的 35 万个片段的子集，因此据我们所知，Octo 是在迄今为止最大的机器人操作演示数据集上训练的。
A key ingredient for training generalist robot policies is robot training data. In contrast to vision and language data that can be scraped from the web, obtaining robot data at scale is challenging and often involves significant investments in hardware and human labor. There are multiple large robot navigation and autonomous driving datasets. In recent years, there have also been multiple efforts for building robot manipulation datasets of increasing scale and diversity, either collected via scripted and autonomous policies or human teleoperation. Octo is trained on the Open X-Embodiment dataset, a recent effort that pooled many of these aforementioned robot datasets. The Open-X dataset contains approximately 1.5M robot episodes, of which we curate 800k for Octo training. We note that the RT-X model used a more restricted subset of 350K episodes, so to the best of our knowledge, Octo is trained on the largest robotics manipulation demonstration dataset to date.
在本节中，我们介绍 Octo 模型，这是一个开源的通用机器人策略，可通过微调适应新的机器人和任务——包括新的感官输入和动作空间。我们讨论了关键设计决策、训练目标、训练数据集和基础设施。Octo 模型的设计强调灵活性和规模：它支持多种常用机器人、传感器配置和动作，同时提供一种通用且可扩展的方案，可以在大量数据上进行训练。它还支持自然语言指令、目标图像、观测历史以及通过扩散解码进行多模态、分块的动作预测。此外，我们专门设计了 Octo 以支持对新机器人设置的高效微调，包括具有不同动作空间和不同相机与本体感觉信息组合的机器人。这一设计旨在使 Octo 成为一个灵活且广泛适用的通用机器人策略，可用于各种下游机器人应用和研究项目。
In this section, we describe the Octo model, our open-source generalist robot policy that can be adapted to new robots and tasks — including new sensory inputs and action spaces — via fine-tuning. We discuss the key design decisions, training objectives, training dataset, and infrastructure. The design of the Octo model emphasizes flexibility and scale: it supports a variety of commonly used robots, sensor configurations, and actions while providing a generic and scalable recipe that can be trained on large amounts of data. It also supports natural language instructions, goal images, observation histories, and multi-modal, chunked action prediction via diffusion decoding. Furthermore, we designed Octo specifically to enable efficient fine-tuning to new robot setups, including robots with different action spaces and different combinations of cameras and proprioceptive information. This design was selected to make Octo a flexible and broadly applicable generalist robot policy that can be utilized for a variety of downstream robotics applications and research projects.
Octo 的核心是一个基于 Transformer 的策略 \(\pi\)。它由三个关键部分组成：输入分词器，将语言指令 \(\ell\)、目标 \(g\) 和观测序列 \(o_{1},\dots,o_{H}\) 转换为 token \(\big[\mathcal{T}_{l},\mathcal{T}_{g},\mathcal{T}_{o}\big]\)（图 0 左）；Transformer 主干网络，处理这些 token 并生成嵌入 \(e_{l},e_{g},e_{o}=T(\mathcal{T}_{l},\mathcal{T}_{g},\mathcal{T}_{o})\)（图 0 上）；以及读出头 \(R(e)\)，产生期望的输出，即动作 \(a\)。
At its core, Octo is a transformer-based policy \(\pi\). It consists of three key parts: input tokenizers that transform language instructions \(\ell\), goals \(g\), and observation sequences \(o_{1},\dots,o_{H}\) into tokens \(\big[\mathcal{T}_{l},\mathcal{T}_{g},\mathcal{T}_{o}\big]\) (Fig. 0, left); a transformer backbone that processes the tokens and produces embeddings \(e_{l},e_{g},e_{o}=T(\mathcal{T}_{l},\mathcal{T}_{g},\mathcal{T}_{o})\) (Fig. 0, top); and readout heads \(R(e)\) that produce the desired outputs, i.e., actions \(a\) .
我们将任务定义（例如语言指令 \(\ell\) 和目标图像 \(g\)）以及观测 \(o\)（例如腕部相机和第三人称相机流）通过特定模态的分词器转换为通用的“分词”格式（见图 0 左）：
We convert task definitions (e.g., language instructions \(\ell\) and goal images \(g\)) and observations \(o\) (e.g., wrist and third-person camera streams) into a common “tokenized” format using modality-specific tokenizers (see Fig. 0, left):
语言输入经过分词后，通过一个预训练的 Transformer 模型，生成一系列语言嵌入向量。我们使用 t5-base（111M）模型。
Language inputs are tokenized, then passed through a pretrained transformer that produces a sequence of language embedding tokens. We use the t5-base (111M) model.
图像观测和目标图像通过一个浅层卷积网络，然后分割为一系列展平的图像块。
Image observations and goals are passed through a shallow convolution stack, then split into a sequence of flattened patches.
我们通过将可学习的位置嵌入 \(p\) 添加到任务和观测向量上，并按顺序排列它们 \(\big[\mathcal{T}_{T},\mathcal{T}_{o,1},\mathcal{T}_{o,2},\dots\big]\) 来组装 Transformer 的输入序列。
We assemble the input sequence of the transformer by adding learnable position embeddings \(p\) to task and observation tokens and then arranging them sequentially \(\big[\mathcal{T}_{T},\mathcal{T}_{o,1},\mathcal{T}_{o,2},\dots\big]\) .
一旦输入被转换为统一的词元序列，它们将由 Transformer 处理（见图 0 顶部）。这与先前在观测和动作序列上训练基于 Transformer 的策略的工作类似。Octo 的注意力模式采用分块掩码：观测词元只能因果地关注来自相同或更早时间步 \(\mathcal{T}_{o,0:t}\) 的词元以及任务词元 \(\mathcal{T}_{T}\)（绿色）。对于不存在的观测对应的词元（例如，没有语言指令的数据集）会被完全掩码。这种模块化设计使我们能够在微调期间添加或移除观测或任务（见下文）。除了这些输入词元块之外，我们还插入了可学习的读出词元 \(\mathcal{T}_{R,t}\)（紫色）。在 \(\mathcal{T}_{R,t}\) 处的读出词元关注序列中其之前的观测和任务词元，但任何观测或任务词元都不会关注它——因此，它们只能被动地读取和处理内部嵌入，而不影响它们。读出词元的作用类似于 BERT 中的 [CLS] 词元，作为迄今为止观测序列的紧凑向量嵌入。一个实现扩散过程的轻量级“动作头”应用于读出词元的嵌入。该动作头预测多个连续动作的“块”，类似于先前的工作。
Once the inputs have been cast to a unified token sequence, they are processed by a transformer (see Fig. 0, top). This is similar to prior works that train transformer-based policies on sequences of observations and actions. The attention pattern of the Octo transformer is block-wise masked: observation tokens can only attend causally to tokens from the same or earlier time steps \(\mathcal{T}_{o,0:t}\) as well as task tokens \(\mathcal{T}_{T}\) (green). Tokens corresponding to non-existing observations are fully masked out (e.g., a dataset without language instructions). This modular design enables us to add and remove observations or tasks during finetuning (see below). In addition to these input token blocks, we insert learned readout tokens \(\mathcal{T}_{R,t}\) (purple). A readout token at \(\mathcal{T}_{R,t}\) attends to observation and task tokens before it in the sequence, but is not attended to by any observation or task token — hence, they can only passively read and process internal embeddings without influencing them. Readout tokens act similarly to the [CLS] token in BERT, serving as a compact vector embedding of the observation sequence thus far. A lightweight “action head” that implements the diffusion process is applied to the embeddings for the readout tokens. This action head predicts a “chunk" of several consecutive actions, similar to prior work.
我们的设计允许我们在下游微调期间灵活地向模型添加新的任务和观测输入或动作输出头。当在下游添加新任务、观测或损失函数时，我们可以完全保留 Transformer 的预训练权重，仅根据规格变化的需要添加新的位置嵌入、新的轻量级编码器或新头的参数（见图 0 底部）。这与先前的架构形成对比，在那些架构中，添加或移除图像输入或更改任务规格将需要重新初始化或重新训练预训练模型的大型组件。
Our design allows us to flexibly add new task and observation inputs or action output heads to the model during downstream finetuning. When adding new tasks, observations, or loss functions downstream, we can wholly retain the pretrained weights for the transformer, only adding new positional embeddings, a new lightweight encoder, or the parameters of the new head as necessitated by the change in specification (see Fig. 0, bottom). This is in contrast to prior architectures, where adding or removing an image input or changing the task specification would require re-initializing or re-training large components of the pre-trained model.
这种灵活性对于使 Octo 成为真正的“通用”模型至关重要：由于我们无法在预训练期间覆盖所有可能的机器人传感器和动作配置，因此在微调期间能够调整 Octo 的输入和输出使其成为机器人社区的多功能工具。先前的模型设计使用标准 Transformer 主干或将视觉编码器与 MLP 输出头融合，锁定了模型期望的输入类型和顺序。相比之下，切换 Octo 的观测或任务不需要重新初始化大部分模型。
This flexibility is crucial to make Octo a truly “generalist” model: since we cannot cover all possible robot sensor and action configurations during pretraining, being able to adapt Octo’s inputs and outputs during finetuning makes it a versatile tool for the robotics community. Prior model designs that use standard transformer backbones or fuse visual encoders with MLP output heads lock in the type and order of inputs expected by the model. In contrast, switching the observation or task for Octo does not require re-initializing most of the model.
我们在来自 Open X-Embodiment 数据集的 25 个数据集的混合上训练 Octo，该数据集是一个多样化的机器人学习数据集集合。我们的训练混合包括来自多种机器人形态和场景的各种任务的演示数据。这些数据集不仅在机器人类型上异构，而且在传感器（例如，是否包含腕部相机）和标签（例如，是否包含语言指令）上也是异构的。详细的混合比例见图 1 和附录 C。为了构建我们的训练混合 \(D\)，我们首先剔除所有不包含图像流的 Open-X 数据集，以及那些不使用增量末端执行器控制的数据集。我们还剔除了过于重复、图像分辨率低或任务过于小众的数据集。对于剩余的数据集，我们根据任务和环境将其大致分为“更多样化”和“较少多样化”两类，并在训练中将更多样化数据集的权重加倍。我们还降低了一些包含大量重复片段的数据集的权重，以避免它们主导混合。最后，我们对缺失的相机通道进行零填充，并对齐数据集之间的夹爪动作空间，使得夹爪命令 +1 表示“夹爪打开”，0 表示“夹爪关闭”。虽然我们发现由此产生的训练混合效果良好，但未来的工作应对预训练通用机器人策略的数据混合质量进行更彻底的分析。
We train Octo on a mixture of 25 datasets from the Open X-Embodiment Dataset, a diverse collection of robot learning datasets. Our training mixture includes demonstration data of a variety of tasks from several robot embodiments and scenes. These datasets are heterogeneous not just in terms of the robot type, but also in the sensors (e.g., including or not including wrist cameras) and labels (e.g., including or not including language instructions). See Fig. 1 and Appendix C for the detailed mixture. To create our training mixture \(D\), we curate the data by first removing all Open-X datasets that contain no image streams, as well as those that do not use delta end-effector control. We also remove datasets that are too repetitive, have a low image resolution, or consist of excessively niche tasks. For the remaining datasets, we roughly categorize them into “more diverse” and “less diverse” datasets based on the tasks and environments, and then double the weight of the more diverse datasets during training. We also down-weight a few datasets with many repetitive episodes to avoid dominating the mixture. Finally, we zero-pad any missing camera channels and align the gripper action spaces between the datasets such that a gripper command of +1 means “the gripper is open” and 0 means “the gripper is closed.” While we found the resulting training mixture to work well, future work should perform a more thorough analysis of data mixture quality for pretraining general robot policies.
我们使用条件扩散解码头来预测连续、多模态的动作分布。重要的是，每次动作预测仅对 Transformer 主干网络执行一次前向传播，之后多步去噪过程完全在小型扩散头内进行。我们发现，这种策略参数化在零样本和微调评估中均优于使用 MSE 动作头或离散化动作分布训练的策略。为了生成动作，我们采样一个高斯噪声向量 \(x^{K}\sim\mathcal{N}\big(0,I\big)\)，并应用 \(K\) 步去噪，使用一个学习到的去噪网络 \(\epsilon_{\theta}(x^{k},e,k)\)，该网络以先前去噪步骤的输出 \(x^{k}\)、步骤索引 \(k\) 以及 Transformer 动作读头的输出嵌入 \(e\) 为条件。
We use a conditional diffusion decoding head to predict continuous, multi-modal action distributions. Importantly, only one forward pass of the transformer backbone is performed per action prediction, after which the multi-step denoising process is carried out entirely within the small diffusion head. We found this policy parameterization to outperform policies trained with MSE action heads or discretized action distributions in both zero-shot and finetuning evaluations. To generate an action, we sample a Gaussian noise vector \(x^{K}\sim\mathcal{N}\big(0,I\big)\) and apply \(K\) steps of denoising with a learned denoising network \(\epsilon_{\theta}(x^{k},e,k)\) that is conditioned on the output \(x^{k}\) of the previous denoising step, the step index \(k\), and the output embedding \(e\) of the transformer action readout.
超参数 \(\alpha\)、\(\gamma\) 和 \(\sigma\) 对应噪声调度：我们使用标准的余弦调度（来自文献）。我们使用标准 DDPM 目标（首次提出于文献）训练扩散头，其中我们向数据集动作添加高斯噪声，并训练去噪网络 \(\epsilon_{\theta}(x^{k},e,k)\) 以重建原始动作。关于扩散策略训练的详细说明，请参见文献。我们在附录 D 中列出了所有超参数。
The hyperparameters \(\alpha\), \(\gamma\), and \(\sigma\) correspond to the noise schedule: we use the standard cosine schedule from . We train the diffusion head using the standard DDPM objective first proposed in , where we add Gaussian noise to the dataset actions and train the denoising network \(\epsilon_{\theta}(x^{k},e,k)\) to reconstruct the original action. For a detailed explanation of diffusion policy training, see . We list all hyperparameters in Appendix D.
在微调期间，我们使用相同的扩散训练目标并更新整个模型，这一方案优于冻结预训练参数子集的方案。在所有微调实验中，我们采用相同的方案：给定一个包含约 100 条轨迹的小型目标域数据集，我们使用余弦衰减学习率（带线性预热）微调 50k 步。
We use the same diffusion training objective during finetuning and update the full model, a recipe which outperformed those that freeze subsets of the pretrained parameters. In all finetuning experiments, we employ the same recipe: given a small target domain dataset with around 100 trajectories, we finetune for 50k steps using a cosine decay learning rate decay with linear warmup.
我们训练了两种模型变体：Octo-Small，其 Transformer 主干与 ViT-S 大小相当；以及 Octo-Base，其 Transformer 主干与 ViT-B 大小相当。
We trained two variants of our model: Octo-Small with a transformer backbone that mirrors the size of a ViT-S, and Octo-Base with a transformer backbone that mirrors the size of a ViT-B.
我们使用 AdamW 优化器，采用逆平方根衰减学习率调度，权重衰减为 0.1，梯度裁剪为 1.0。ViT-B 在 TPU v4-128 pod 上训练了 300k 步，批大小为 2048，耗时 14 小时。同一模型在单块 NVIDIA A5000 GPU（24GB 显存）上进行微调运行大约需要 5 小时，并且可以通过多 GPU 训练加速。
We use the AdamW optimizer with an inverse square root decay learning rate schedule, with weight decay of 0.1 and gradient clipping of 1.0. The ViT-B was trained for 300k steps with a batch size of 2048 using a TPU v4-128 pod, which took 14 hours. A finetuning run of the same model on a single NVIDIA A5000 GPU with 24GB of VRAM takes approximately 5 hours and can be sped up with multi-GPU training.
我们使用 2 帧观测历史进行训练；在初步实验中，我们发现超过第一额外帧后收益显著递减。我们采用事后目标重标注（hindsight goal relabeling），即从轨迹的未来中均匀选择一个状态作为目标图像，这与先前工作类似。我们在训练中应用常见的图像数据增强，并随机将每个训练样本的语言指令或目标图像置零，以使 Octo 能够基于语言指令或目标图像进行条件生成。对于没有语言标注的数据集，我们始终使用目标图像条件。这使得我们的模型主要从自监督视觉观测中学习控制，并减少了对语言标注的依赖，这与先前关于多上下文模仿学习的工作类似。关于超参数选择的更多细节，请参见附录 D。
We train using 2 frames of observation history; in our preliminary experiments, we found significantly diminishing gains beyond the first additional frame. We use hindsight goal relabeling, which selects a state uniformly from the future in the trajectory to assign as the goal image, similar to prior work. We apply common image data augmentations during training, and randomly zero out the language instruction or goal image per training example to enable Octo to be conditioned on either language instructions or goal images. For datasets without language annotations, we always use goal image conditioning. This enables our model to learn control mostly from self-supervised visual observations and reduces the burden on language annotation, similar to prior work on multi-context imitation learning. For more details on the choice of hyperparameters, see Appendix D.
我们开源了训练、微调和运行模型所需的全部资源（参见 https://octo-models.github.io）：
We open-source all resources required to train, finetune and run our model (see https://octo-models.github.io):
Octo-Small（2700 万参数）和 Octo-Base（9300 万参数）的预训练 Octo 检查点。
Pretrained Octo checkpoints for Octo-Small (27M params) and Octo-Base (93M params).
用于 Octo 模型的微调脚本，基于 JAX。
Finetuning scripts for Octo models, in JAX.
在 Open X-Embodiment 数据集上进行 Octo 预训练的模型预训练流程，基于 JAX。
Model pretraining pipeline for Octo pretraining on the Open X-Embodiment dataset, in JAX.
用于 Open X-Embodiment 数据的独立数据加载器，兼容 JAX 和 PyTorch。
Standalone data loaders for Open X-Embodiment data, compatible with JAX and PyTorch.
我们在附录 B 中提供了一个加载和运行预训练 Octo 模型的简单示例。
We provide a simple example for loading and running a pretrained Octo model in Appendix B.
我们的实验对 Octo 进行了实证分析，评估其作为通用机器人基础模型的能力，涵盖以下几个维度：
Our experiments provide an empirical analysis of Octo, evaluating its ability to serve as a general robotic foundation model across several axes:
Octo 能否开箱即用地控制多种机器人形态，并解决语言和目标任务？
Can Octo control multiple robot embodiments and solve language and goal tasks out of the box?
Octo 的权重能否作为数据高效微调至新任务和新机器人的良好初始化，并且是否优于从头训练和常用的预训练表示？
Do Octo weights serve as a good initialization for data-efficient finetuning to new tasks and robots, and does it improve over training from scratch and commonly used pretrained representations?
在构建通用机器人策略时，Octo 的哪些设计决策最为关键？
Which design decisions in Octo matter most for building generalist robot policies?
我们在 4 个机构的 9 种代表性机器人学习设置上评估 Octo 的能力（见图 2）。我们测试 Octo 在匹配预训练数据的机器人设置上，对语言和目标图像任务进行开箱即用（“零样本”）控制不同机器人的能力，其中所有机器人均采用增量末端执行器控制动作，观测空间为 RGB 图像。我们还评估了 Octo 在新环境和任务上的数据高效微调能力，包括新观测（“Berkeley Insertion”中的力-扭矩输入）、新动作空间（“Berkeley Pick-Up”中的关节位置控制）和新机器人本体（“Berkeley Coke”和“Berkeley Bimanual”）。每个微调设置使用约 100 个域内演示，并在 NVIDIA A5000 GPU 上微调不到 5 小时，所有设置使用相同的超参数（见附录 D）。我们的评估任务测试 Octo 与多样物体交互（如“WidowX BridgeV2”）、解决长时程任务（如“Stanford Coffee”）以及执行精细操作（如“Berkeley Insertion”）的能力。有关每个评估设置的更多细节，请参见附录 F。
We evaluate Octo's capabilities across a representative spectrum of 9 robot learning setups at 4 institutions (see Fig. 2). We test Octo's ability to control different robots out-of-the-box (“zero-shot”) for language and goal image tasks using robot setups that match the pretraining data, where all robots are controlled with delta end-effector control actions and the observation spaces are RGB images. We also evaluate Octo for data-efficient finetuning to new environments and tasks, including with new observations (force-torque inputs in “Berkeley Insertion”), new action spaces (joint position control in “Berkeley Pick-Up”) and new robot embodiments (“Berkeley Coke” and “Berkeley Bimanual”). Each of the finetuning setups uses \(\sim 100\) in-domain demonstrations and finetunes in \(<5\) hours on a NVIDIA A5000 GPU, using the same hyperparameters across all setups (see Appendix D). Our evaluation tasks test Octo's ability to interact with diverse objects (e.g., “WidowX BridgeV2”), solve long-horizon tasks (e.g., “Stanford Coffee”) and perform precise manipulation (e.g., “Berkeley Insertion”). For more details on each evaluation setup, see Appendix F.
我们将 Octo 的开箱即用多机器人控制能力与最佳公开可用的通用机器人策略 RT-1-X 进行比较，使用其发布的检查点。与 Octo 类似，RT-1-X 在 Open X-Embodiment 机器人数据集上预训练，旨在零样本控制多个机器人，因此提供了自然的比较点。我们还将 Octo 的零样本能力与 RT-2-X 进行比较，RT-2-X 是一个 550 亿参数的视觉-语言模型，在 Open X-Embodiment 数据集上微调以生成机器人动作。RT-1-X 和 RT-2-X 模型在更受限的 350K 集（相比之下 Octo 为 800K 集）上训练。我们进一步将 Octo 作为数据高效微调的策略初始化的性能与两种常见方法进行比较：(1) 从零开始在目标领域演示上训练，(2) 使用预训练的视觉表示。虽然许多先前工作提出了其他用于模仿微调的预训练方案，但据我们所知，没有先前方法提供已被证明能成功微调到新观察和动作空间的预训练策略。然而，预训练的视觉表示（如 VC-1）已以这种方式使用，因此我们将这些方法作为另一个比较点。
We compare Octo's ability to control multiple robots out-of-the-box to the best openly available generalist robot policy, RT-1-X, using the released checkpoint. Similar to Octo, RT-1-X is pretrained on the Open X-Embodiment robot dataset and aims to control multiple robots zero-shot, thus providing a natural point of comparison. We also compare the zero-shot capabilities of Octo to RT-2-X, a 55 billion parameter vision-language model finetuned on the Open X-Embodiment dataset to produce robot actions. The RT-1-X and RT-2-X models are trained on a more restricted subset of 350K episodes (compared to 800k episodes for Octo). We further compare Octo's performance as a policy initialization for data efficient finetuning to two common approaches: (1) training on the target domain demonstrations from scratch and (2) using pretrained visual representations. While a number of prior works have proposed other pretraining schemes for imitation finetuning, to our knowledge no prior method provides a pretrained policy that has been demonstrated to finetune successfully to new observation and action spaces. However, pretrained visual representations such as VC-1 have been used in this way, and therefore we use these methods as another point of comparison.
对于微调，我们发现从头训练我们的大型 Transformer 架构在小型数据集上会迅速过拟合。相反，我们使用许多先前工作采用的典型策略架构获得了更好的从头训练结果：带有 FiLM 语言条件的 ResNet 视觉编码器，结合以扩散目标训练的小型 Transformer 动作解码器，类似于 。我们的该架构实例有 2800 万参数（类似于 RT-1）。我们将其作为我们的从头训练基线（“ResNet+Transformer 从头训练”）。我们还按照 的过程与预训练视觉表示进行比较。ViT-B 视觉编码器初始化为 VC-1 权重，这是一种最先进的视觉表示，在 4000 小时的自我中心视频和 ImageNet 上预训练，并与 MLP 动作解码器结合。整个模型使用 MSE 损失训练以预测专家动作（“VC-1”）。
For finetuning, we found that training our large transformer architecture from scratch overfit quickly on the small datasets. Instead, we obtained better from-scratch results using a canonical policy architecture employed by many prior works: a ResNet visual encoder with FiLM language conditioning, combined with a small transformer action decoder trained with a diffusion objective, similar to . Our instantiation of this architecture has 28M parameters (similar to RT-1). We adopt this as our from-scratch baseline ("ResNet+Transformer Scratch"). We also compare to a pretrained visual representation following the procedure of . A ViT-B visual encoder is initialized to the VC-1 weights, a state-of-the-art visual representation pretrained on 4,000 hours of ego-centric videos and ImageNet, and combined with an MLP action decoder. The full model is trained to predict expert actions using an MSE loss ("VC-1").
我们在图 3 中比较了 Octo、RT-1-X 和 RT-2-X 的零样本操作能力。我们评估了从预训练数据集中选取的多个任务，包括拾取与放置、用布擦拭桌子以及开关抽屉。对于每台机器人，我们从相应的 OXE 数据集中选择了两个语言任务，并在不同初始条件下对每个任务执行 10 次试验（详见附录 F）。所选任务属于预训练数据中的“分布内”任务，但评估要求方法能够泛化到新的物体位置、光照条件、背景和干扰物体。尽管所有方法在预训练环境中的任务上表现合理，但我们发现 Octo 的平均成功率比 RT-1-X（3500 万参数）高出 29%。在 WidowX 和 RT-1 机器人的评估中，我们还与 RT-2-X（550 亿参数）进行了比较，发现 Octo 的表现与之相当。
We compare the zero-shot manipulation capabilities of Octo, RT-1-X, and RT-2-X in Fig. 3. We evaluated on several tasks selected from the pre-training dataset including picking and placing, wiping a table with a cloth, and opening and closing drawers. For each robot, we selected two language tasks from the corresponding OXE dataset and performed 10 trials per task with varying initial conditions (details in Appendix F). The chosen tasks are “in-distribution” from the pre-training data, but the evaluation requires methods to generalize to new object positions, lighting conditions, backgrounds, and distractor objects. While all methods acted reasonably across tasks in the pretraining environments, we found that on average Octo had a 29% higher success rate than RT-1-X (35M parameters). For the WidowX and RT-1 Robot evaluations, we also compared to RT-2-X (55 billion parameters) and found that Octo performed similarly.
此外，RT-1-X 和 RT-2-X 仅支持语言指令条件，而 Octo 还支持目标图像条件。我们在 WidowX 任务上使用目标图像条件评估了我们的模型，发现其成功率比使用语言条件时高出 25%。这可能是因为目标图像提供了更多关于如何完成任务的信息。在 BridgeV2 领域，我们在表 VII 中对零样本能力进行了细粒度分析，测量了在数据集中见过的设置以及新环境、新场景和新技能上的表现。虽然 Octo 模型在新物体上取得了较高的成功率，但在新场景中零样本性能略有下降，而在翻转或精确插入等新行为上则大幅下降。
Additionally, while RT-1-X and RT-2-X only support conditioning on language instructions, Octo also supports conditioning on goal images. We evaluated our model on the WidowX tasks using goal image conditioning and found that it achieved a 25% higher success rate than when evaluated with language conditioning. This is likely because goal images provide more information about how to achieve the task. In the BridgeV2 domain, we performed a fine-grained analysis of the zero-shot capabilities in Table VII; measuring performance on setups seen in the dataset, and for novel environments, scenes, and skills. While the Octo model achieves high success on novel objects, zero-shot performance slightly degrades in a new scene, and high degradation for novel behaviors like flipping or precise insertion.
我们在表 I 中报告了在新领域进行数据高效微调的结果。我们发现，微调 Octo 比从头开始训练或使用预训练的 VC-1 权重能获得更好的策略。在六个评估设置（详见附录 F）的平均结果中，Octo 比次优基线高出 52%。重要的是，我们在所有评估任务上使用相同的配方和超参数来微调 Octo（见第 III-C 节），这使得它成为一个良好的默认配置。
We report data-efficient fine-tuning results to new domains in Table I. We find that fine-tuning Octo leads to better policies than starting from scratch or with the pretrained VC-1 weights. On average across the six evaluation setups (detailed in Appendix F), Octo outperforms the next best baseline by 52%. Importantly, we use the same recipe and hyperparameters for fine-tuning Octo on all evaluation tasks (see Section III-C), making this a good default configuration.
结果还强调了 Octo 适应新观测（“Berkeley Insertion”的力-扭矩输入）、动作空间（“Berkeley Pick-Up”的关节位置控制）和新机器人本体（“Berkeley Coke”和“Berkeley Bimanual”）的能力。这使得 Octo 适用于广泛的单臂和双臂机器人操作问题，而这些问题超出了单一相机输入和末端执行器位置控制的范围。
The results also underline Octo's ability to accommodate new observations (force-torque inputs for "Berkeley Insertion"), action spaces (joint position control for "Berkeley Pick-Up") and new robot embodiments ("Berkeley Coke" and "Berkeley Bimanual"). This makes Octo applicable to a wide range of single and dual arm robotic manipulation problems that go beyond a single camera input and end-effector position control.
我们已经证明了 Octo 作为零样本多机器人控制器和策略微调初始化的有效性。接下来，我们分析不同设计决策对 Octo 策略性能的影响。具体而言，我们关注以下几个方面：（1）模型架构，（2）训练数据，（3）训练目标，以及（4）模型规模。除非另有说明，由于我们的算力预算，我们所有消融实验均在 Octo-Small 模型上进行。
We have demonstrated the effectiveness of Octo as a zero-shot multi-robot controller and as an initialization for policy finetuning. We next analyze the effects of different design decisions on the performance of the Octo policy. Concretely, we focus on the following aspects: (1) model architecture, (2) training data, (3) training objective, and (4) model scale. Unless noted otherwise, we perform all ablations on the Octo-Small model due to our compute budget.
先前的基于 Transformer 的策略设计通常使用大型 ResNet 风格的编码器对输入图像进行编码，并将得到的图像特征与一个相对较小的 Transformer 融合。相反，我们选择了一种“Transformer 优先”的架构，该架构使用非常浅的 CNN 补丁编码器，并将大部分参数和 FLOPS 集中在 Transformer 主干中，类似于经典的视觉 Transformer 架构。在表 II 中，我们展示了这种可扩展的架构在完整 Open X-Embodiment 数据混合上训练时能显著提升性能。重要的是，我们发现基于 ResNet 的架构在小数据集上训练时（例如，在我们的“从头开始”比较中）表现优于 ViT，这强调了大型 Transformer 策略特别适合在多样化数据集上进行可扩展训练。
Prior transformer-based policy designs typically encode input images with large ResNet-style encoders and fuse the resulting image features with a comparatively small transformer. Instead, we opt for a “transformer-first” architecture that uses very shallow CNN patch encoders and concentrates most of the parameters and FLOPS in the transformer backbone, similar to canonical vision transformer architectures. In Table II we show that this scalable architecture leads to substantially improved performance when training on the full Open X-Embodiment data mix. Importantly, we found ResNet-based architectures to perform better than ViTs when training on small datasets, e.g., in our “from scratch” comparisons, underlining that large transformer policies are uniquely suited for scalable training on diverse datasets.
Octo 在迄今为止最多样化的跨具身机器人数据集上训练，该数据集由我们从 Open X-Embodiment 数据集中手动整理的 25 个数据集混合而成（见第 III-B 节）。我们通过将 Octo 模型与使用 RT-X 模型训练中使用的 11 个数据集的较小混合训练的 Octo 模型以及仅在目标机器人领域数据上训练的基线进行比较，来消融这种训练混合的影响。在表 II 中，我们展示了随着训练数据集数量的增加，Octo 的性能也随之提升。这表明将数据混合扩展到更多数据集可能进一步提高策略性能。我们将此留作未来工作，同时更深入地研究数据整理的最佳实践。
Octo is trained on the most diverse cross-embodied robot dataset to date, a mix of 25 datasets that we manually curated from the Open X-Embodiment dataset (see Section III-B). We ablate the impact of this training mix by comparing to Octo models trained on a smaller mix of 11 datasets used in training the RT-X models and a baseline trained only on data from the target robot domain. In Table II we show that the performance of Octo increases as we increase the number of training datasets. This suggests that expanding the data mix to even more datasets may further improve policy performance. We will leave this for future work, along with a more thorough investigation of best practices for data curation.
我们将 Octo 的扩散解码训练目标（见第 III-C 节）与先前工作中常见的替代方案进行比较：简单的 MSE 损失和离散化动作上的交叉熵损失。在表 II 中，我们发现 Octo 的扩散训练目标显著提升了性能。这种提升很可能是因为扩散头能够建模多模态动作分布（不同于 MSE 头），同时保留了连续动作的精度（不同于离散头）。从定性上看，该策略比 MSE 训练的策略行动更果断，比离散化动作训练的策略行动更精确。
We compare Octo's diffusion decoding training objective (see Section III-C) to common alternatives from prior work: simple MSE loss and cross-entropy loss on discretized actions. In Table II we find that Octo's diffusion training objective leads to substantially improved performance. This improvement is likely because the diffusion head can model multi-modal action distributions (unlike the MSE head) while retaining the precision of continuous actions (unlike the discrete head). Qualitatively, the policy acts more decisively than MSE-trained policies, and more precisely than those trained with discretized actions.
我们按照常见视觉 Transformer 模型的阶梯，比较了三种不同规模的 Octo 模型：Octo-Tiny（10M）、Octo-Small（27M）和 Octo-Base（93M）。在图 4 中，我们展示了策略的零样本性能随模型规模增大而提升。我们发现，Base 模型对初始场景配置的鲁棒性优于 Small 模型，且更不容易过早尝试抓取，这表明更大的模型具有更好的视觉场景感知能力。
We compare Octo models of three different sizes following the ladder of common vision transformer models: Octo-Tiny (10M), Octo-Small (27M), and Octo-Base (93M). In Figure 4, we show that the zero-shot performance of the policy scales with increasing model size. We find that the Base model is more robust to initial scene configuration than the Small model, and is less prone to early grasp attempts, indicating that the larger model has better visual scene perception.
我们介绍了 Octo，这是一个基于 Transformer 的大型策略，在迄今为止最大的机器人操作数据集（80 万条机器人轨迹）上进行了预训练。我们证明了 Octo 能够开箱即用地解决各种任务，并展示了 Octo 的组合式设计如何使其能够微调以适应新的输入和动作空间，从而使 Octo 成为各种机器人控制问题的通用初始化。除了模型本身，我们还发布了完整的训练和微调代码，以及便于在大型机器人数据集上训练的工具。
We introduced Octo, a large transformer-based policy pretrained on the largest robot manipulation dataset to date, 800k robot trajectories. We demonstrated that Octo can solve a variety of tasks out-of-the-box and showed how Octo’s compositional design enables finetuning to new inputs and action spaces, making Octo a versatile initialization for a wide range of robotic control problems. Apart from the model itself, we have released our full training and finetuning code, alongside tools that make it easier to train on large robot datasets.
尽管 Octo 在零样本和微调评估中都取得了强劲的性能，但我们发现当前模型仍存在一些不足，这在很大程度上归因于训练数据的特性。首先，我们发现当前的 Octo 模型在处理腕部相机信息方面存在困难。通常，仅使用第三人称相机而非结合第三人称和腕部相机时，微调结果更强。此外，我们注意到语言条件策略与目标条件策略的性能之间存在较大差异。在这两种情况下，训练数据中相应模态的缺乏可能是原因：只有 27% 的数据包含腕部相机信息，只有 56% 的预训练数据包含语言标注。
While Octo achieves strong performance in both zero-shot and finetuning evaluations, we find that the current model still has several shortcomings, which we attribute in large parts to characteristics of the training data. First, we found that the current Octo model struggles with adequately processing wrist camera information. Often finetuning results were stronger when using only a third person camera instead of combining third person and wrist camera. Additionally, we notice a large difference between language-conditioned policy performance and goal-conditioned policy performance. In both cases, a lack of the respective modalities in the training data is the likely reason: only 27% of the data contains wrist camera information and only 56% of the pretraining data contains language annotations.
扩展用于训练 Octo 的数据是一个自然的改进途径。由于 Open X-Embodiment 数据集由最优机器人演示组成，当前模型通过模仿进行训练；未来的工作可以考虑从次优或在线交互数据中学习，这需要替代目标。此外，虽然我们仅在单臂和双臂操作器上训练和评估了 Octo，但扩展到更广泛的机器人（如执行导航或移动操作）将是一个高机遇的方向。
Expanding the data used to train Octo is a natural avenue of improvement. Since the Open X-Embodiment dataset is comprised of optimal robot demonstrations, the current model trains via imitation; future work may consider learning from sub-optimal or online interaction data that require alternative objectives. Further, while we trained and evaluated Octo exclusively on single and dual-arm manipulators; expanding to a wider set of robots that perform navigation or mobile manipulation would be a direction of high opportunity.
虽然 Octo 朝着构建能在各种机器人设置上开箱即用的通用机器人策略迈出了一步，但仍需改进模型，包括更好的语言条件、改进对腕部相机的支持，以及纳入超越最优演示的数据。我们希望 Octo 能为研究人员和从业者提供一个简单的起点，以便访问更大的机器人数据集，并利用预训练的机器人模型高效学习新任务和实现广泛泛化。
While Octo represents a step towards building generalist robot policies that work out-of-the-box on diverse robot setups, there remains work to improve the model, including better language conditioning, improved support for wrist cameras, and incorporating data beyond optimal demonstrations. We hope that Octo offers a simple launchpad for researchers and practitioners to access larger robotic datasets and leverage pretrained robotics models for efficient learning of new tasks and broad generalization.