NVIDIA introduced GR00T 2 at the CoRL conference, an open model designed to teach robots dexterous manipulation. Built on NVIDIA Cosmos, the model learns from human demonstrations and can transfer skills between different robot bodies without robot demonstrations, according to the company.
Learning from demonstrations and test results
GR00T 2 is a world action model. NVIDIA says it trained the model on diverse robot and human data, including 100,000 hours of its open human-viewpoint video recordings.
In an evaluation, the model completed 40 of 50 tasks at least once without a demonstration specific to each task, according to the company. With one demonstration per task, it achieved an average success rate of 71% across the 50 tasks; with five demonstrations, the rate rose to 95%.
NVIDIA also says the model ranks among the top three in every vision-language-action and world action category across the RoboCasa365, RoboLab, RoboDojo and MolmoSpaces benchmarks. According to the company, it leads several of the leaderboards.
AI agents can use the model’s API and related tools to sequence learned skills and recover from errors.
From handling laundry to swapping cables
Nexuni is testing the model for robotic laundry handling, Foxconn is using it to automate server manufacturing, and Siemens is evaluating it for industrial use.
MicroAGI is using GR00T 2, post-trained with reinforcement learning, to swap Ethernet cables with a YAM robotic arm. Its workflow uses Isaac Lab 3.0 and Newton to automatically create and calibrate simulation environments in which the robot policy is trained. A demonstration of the workflow is planned for CoRL.
Video and simulation for robot training
Alongside the model, NVIDIA introduced Video to Data, an open workflow that converts human video into simulated robot demonstrations for creating training data.
The new version of the open Isaac Lab 3.0 framework lets agents prepare training environments and researchers choose between the NVIDIA PhysX and Newton physics solvers. It also includes a new library for post-training foundation models such as GR00T with reinforcement learning.
Tighter integration with Newton, an open-source physics engine, lets Isaac Lab simulate deformable objects such as cables and cloth, as well as complex articulated systems such as robotic hands.
In a vision-based dexterous manipulation benchmark, Isaac Lab 3.0 with Newton achieved up to 2.5 times the training throughput of Isaac Lab 2.3 with PhysX, according to NVIDIA.
Evaluation under different conditions
The developer preview of Isaac Lab-Arena 0.4 lets developers compose repeatable evaluations of robot policies and run them in parallel across thousands of environment variations. Analyses then show which conditions lead to failures.
