Bridging the 'reality gap' that separates simulated robotics from experiments on hardware could accelerate robotic research through improved data availability. This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator. With enough variability in the simulator, the real world may appear to the model as just another variation. We focus on the task of object localization, which is a stepping stone to general robotic manipulation skills. We find that it is possible to train a real-world object detector that is accurate to $1.5$cm and robust to distractors and partial occlusions using only data from a simulator with non-realistic random textures. To demonstrate the capabilities of our detectors, we show they can be used to perform grasping in a cluttered environment. To our knowledge, this is the first successful transfer of a deep neural network trained only on simulated RGB images (without pre-training on real images) to the real world for the purpose of robotic control.
核心贡献 · Key contributions
首次成功将仅使用模拟 RGB 图像训练的深度神经网络迁移到真实机器人控制。 First successful transfer of deep neural network trained only on simulated RGB images to real-world robotic control.
域随机化使真实世界物体定位精度达到 1.5 厘米。 Domain randomization enables object localization accurate to 1.5 cm in the real world.
非真实随机纹理足以实现模拟到真实的迁移，无需在真实图像上预训练。 Non-realistic random textures suffice for sim-to-real transfer without pre-training on real images.
检测器对干扰物和部分遮挡具有鲁棒性，可在杂乱环境中执行抓取。 Detectors robust to distractors and partial occlusions, enabling grasping in cluttered environments.
消融研究表明纹理和相机位姿的随机化对迁移成功至关重要。 Ablation study shows randomization of textures and camera pose critical for transfer success.
方法泛化到具有非均匀纹理的真实物体（如 YCB 物体）。 Method generalizes to real-world objects with non-uniform textures (e.g., YCB objects).