Datasets

Curated datasets for training and developing robotics applications.

BOP Challenge Benchmark

Benchmark for 6D object pose estimation. Includes multiple datasets and evaluation protocols.

Format:Benchmark Suite
License:Various
Stage:
annotated
franka-panda
pose_estimation
perception

ARC X Robotics Benchmark

Amazon's object picking benchmark for robotic manipulation. Standardized evaluation suite.

Format:Benchmark Suite
License:Apache-2.0
Stage:
annotated
franka-panda
manipulation
picking

ACT Model Zoo

Pretrained Action Chunking Transformer models for manipulation. Includes model checkpoints and evaluation code.

Format:Model Weights + Code
License:MIT
Stage:
skill abstracted
widowx
franka-panda
manipulation

ROS Industrial Repository

Robot models, controllers, and configurations for dozens of industrial robot arms. Essential for simulation and deployment.

Format:URDF/Code
License:Various (mostly BSD/MIT)
Stage:
skill adjacent
ur5
kuka-iiwa
franka-panda

LM-O (Linemod-Occluded)

6DoF object pose dataset with 8 household objects. Includes occluded scenarios for robust pose estimation.

Format:RGB-D Images + Poses
License:BOP Terms of Use
Stage:
annotated
franka-panda
pose_estimation
perception

T-LESS

6DoF object pose estimation dataset with 30 industrial objects. Used for manipulation and pose estimation benchmarks.

Format:RGB-D Images + Poses
License:T-LESS Terms of Use
Stage:
annotated
franka-panda
pose_estimation
perception

LIBERO Language-Prompt Datasets

Household manipulation tasks combined with language prompts and demonstration episodes. Language-conditioned skill learning.

Format:HDF5
License:MIT
Stage:
annotated
franka-panda
manipulation
language_grounded

DexMV / Dex-Hand

Dexterous hand manipulation demonstrations. Includes complex in-hand manipulation tasks.

Format:HDF5
License:MIT
Stage:
pre processed
franka-panda
dexterous_manipulation
hand_manipulation

RoboTurk / R2D2

Crowdsourced robot teleoperation dataset. Large-scale human demonstrations collected via web interface.

Format:HDF5
License:MIT
Stage:
raw
sawyer
pr2
manipulation

BC-Z

Large-scale multi-task robotic manipulation dataset with 24,000 demonstration episodes across 100+ tasks.

Format:RLDS
License:Apache-2.0
Stage:
annotated
widowx
manipulation
multi_task

Meta-World

Popular simulation benchmark for generalist RL research. 50 manipulation tasks with diverse complexity.

Format:Simulation
License:MIT
Stage:
pre processed
sawyer
manipulation
multi_task

Franka Kitchen Dataset

Canonical benchmark for multi-step manipulation with Franka Panda. Kitchen task demonstrations.

Format:HDF5
License:MIT
Stage:
raw
franka-panda
kitchen_tasks
multi_step_tasks

BEHAVIOR-1K

Comprehensive simulation suite with 1,000 household tasks. Includes both simulation and real-world components.

Format:Simulation + Data
License:MIT
Stage:
pre processed
fetch
pr2
household_tasks

SayCan / Commands Dataset

Google's language-conditioned manipulation dataset used before RT-1. Includes natural language commands and robot demonstrations.

Format:RLDS
License:Apache-2.0
Stage:
annotated
fetch
manipulation
language_grounded

Diffusion Policy Model Zoo

Dozens of pretrained diffusion-based policies for tabletop manipulation tasks. Ready-to-use skill models.

Format:Model Weights + Code
License:MIT
Stage:
skill abstracted
franka-panda
widowx
manipulation

OpenVLA

Open Visual Language Action model for robot control. Pretrained VLA model that can be fine-tuned for specific tasks.

Format:Model Weights + Code
License:Apache-2.0
Stage:
skill abstracted
franka-panda
widowx
manipulation

Isaac Sim Assets Library

Robot models, articulation trees, sensors, and full simulation scenes for NVIDIA Isaac Sim. Essential for simulation workflows.

Format:USD/URDF
License:Isaac Sim License
Stage:
simulation ready
franka-panda
ur5
kuka-iiwa

KITTI

Classic robot perception dataset for outdoor autonomous driving. Includes stereo images, LiDAR, GPS/IMU, and object labels.

Format:Images + LiDAR + Labels
License:KITTI Terms of Use
Stage:
raw
franka-panda
navigation
perception

Replica / Habitat Scene Datasets

Photorealistic indoor environments from Facebook AI Research. Used for robot navigation, policy learning, and embodied AI.

Format:3D Scenes + Configs
License:MIT
Stage:
annotated
franka-panda
navigation
simulation

Matterport3D

High-resolution indoor 3D scans of 90 building-scale scenes. Essential for navigation, mapping, and scene understanding.

Format:3D Meshes + Images
License:Matterport3D Terms of Use
Stage:
annotated
franka-panda
navigation
mapping

ScanNet

Large-scale indoor scene dataset with 3D reconstructions, semantic segmentation, and camera trajectories. Essential for navigation and scene understanding.

Format:3D Meshes + Images
License:ScanNet Terms of Use
Stage:
annotated
franka-panda
navigation
scene_understanding

HOPE (Household Object Pose Estimation)

6DoF object pose estimation dataset with 28 household objects. Includes RGB-D images and ground truth poses.

Format:RGB-D Images + Poses
License:MIT
Stage:
annotated
franka-panda
pose_estimation
perception

Google Scanned Objects

Real, high-quality 3D scans of household objects from Google Research. Photorealistic meshes with textures.

Format:3D Models (SDF/URDF)
License:Apache-2.0
Stage:
annotated
franka-panda
object_models
perception

YCB Object & Model Set

Standard object models used across almost all manipulation research. Includes 77 household objects with 3D meshes, textures, and physical properties.

Format:3D Models (OBJ/STL/URDF)
License:CC-BY 4.0
Stage:
annotated
franka-panda
object_models
perception

ManiSkill

Massive simulated manipulation tasks with trajectories, 3D scenes, and pretrained models. Supports ManiSkill 1, 2, and 3.

Format:HDF5
License:MIT
Stage:
pre processed
franka-panda
widowx
manipulation

RLBench

Hundreds of multi-step manipulation tasks with 3D scenes and expert demonstrations for sim-to-real learning.

Format:HDF5
License:MIT
Stage:
annotated
franka-panda
pr2
sawyer

WidowX Demonstrations

Collection of manipulation demonstrations using WidowX robot arm.

Format:HDF5
License:MIT
Stage:
raw
widowx
manipulation
pick_and_place

MimicGen

Large-scale dataset generation system for manipulation, enabling scalable data collection.

Format:HDF5
License:Apache-2.0
Stage:
pre processed
franka-panda
widowx
manipulation

Action Chunking with Transformers (ACT)

Manipulation dataset with action chunking demonstrations for transformer-based learning.

Format:HDF5
License:MIT
Stage:
pre processed
widowx
franka-panda
manipulation

DROID

Large-scale manipulation dataset with 76,000 demonstrations across 100+ tasks.

Format:HDF5
License:MIT
Stage:
raw
franka-panda
manipulation
pick_and_place

RT-1-X

Pre-trained models and datasets for cross-embodiment robot learning, built on Open X-Embodiment.

Format:RLDS
License:Apache-2.0
Stage:
skill abstracted
franka-panda
pr2
ur5

Language-Table

Large-scale language-conditioned manipulation dataset with 600k+ language-annotated demonstrations.

Format:RLDS
License:Apache-2.0
Stage:
annotated
franka-panda
manipulation
language_grounded

Bridge Data V2

Large-scale manipulation dataset with 60,000+ demonstrations across diverse tasks and environments.

Format:HDF5
License:MIT
Stage:
raw
franka-panda
widowx
manipulation

RoboSet

Real-world multi-task dataset collected in kitchen scenes with kinesthetic and teleop demonstrations (28,500 trajectories).

Format:HDF5
Stage:
raw
franka-panda
kitchen_tasks
rgb_camera

Open X-Embodiment

Unified open dataset of 1M+ real robot trajectories across 22 robot embodiments for cross-robot learning.

Format:RLDS
License:Apache-2.0 (software) + CC-BY for data
Stage:
pre processed
franka-panda
pick_and_place
rgb_camera

RoboNet

Large-scale multi-robot learning dataset with over 15 million frames from 7 robot platforms.

Format:HDF5
License:CC-BY 4.0
Stage:
raw
franka-panda

WidowX Demonstrations

Collection of manipulation demonstrations using WidowX robot arm.

Format:HDF5
License:MIT
Stage:
raw
widowx
manipulation
pick_and_place

MimicGen

Large-scale dataset generation system for manipulation, enabling scalable data collection.

Format:HDF5
License:Apache-2.0
Stage:
pre processed
franka-panda
widowx
manipulation

Action Chunking with Transformers (ACT)

Manipulation dataset with action chunking demonstrations for transformer-based learning.

Format:HDF5
License:MIT
Stage:
pre processed
widowx
franka-panda
manipulation

DROID

Large-scale manipulation dataset with 76,000 demonstrations across 100+ tasks.

Format:HDF5
License:MIT
Stage:
raw
franka-panda
manipulation
pick_and_place

RT-1-X

Pre-trained models and datasets for cross-embodiment robot learning, built on Open X-Embodiment.

Format:RLDS
License:Apache-2.0
Stage:
skill abstracted
franka-panda
pr2
ur5

Language-Table

Large-scale language-conditioned manipulation dataset with 600k+ language-annotated demonstrations.

Format:RLDS
License:Apache-2.0
Stage:
annotated
franka-panda
manipulation
language_grounded

Bridge Data V2

Large-scale manipulation dataset with 60,000+ demonstrations across diverse tasks and environments.

Format:HDF5
License:MIT
Stage:
raw
franka-panda
widowx
manipulation

RoboSet

Real-world multi-task dataset collected in kitchen scenes with kinesthetic and teleop demonstrations (28,500 trajectories).

Format:HDF5
Stage:
raw
franka-panda
kitchen_tasks
rgb_camera

Open X-Embodiment

Unified open dataset of 1M+ real robot trajectories across 22 robot embodiments for cross-robot learning.

Format:RLDS
License:Apache-2.0 (software) + CC-BY for data
Stage:
pre processed
franka-panda
pick_and_place
rgb_camera

RoboNet

Large-scale multi-robot learning dataset with over 15 million frames from 7 robot platforms.

Format:HDF5
License:CC-BY 4.0
Stage:
raw
franka-panda