NVIDIA unveiled a suite of new AI tools and announced a partnership with Hugging Face at this week's Conference on Robot Learning in Munich, marking a significant push to make advanced robotics more accessible to developers and researchers.
The collaboration brings together Hugging Face's LeRobot open-source framework with NVIDIA's AI and robotics technology, creating a shared ecosystem where developers can access and build upon each other's work. This move could accelerate progress in manufacturing, healthcare, and logistics automation.
"The robotics community thrives when we build together," says Animesh Garg, assistant professor at Georgia Tech. "By embracing open-source frameworks such as Hugging Face's LeRobot and NVIDIA Isaac Lab, we accelerate the pace of research and innovation in AI-powered robotics."
At the heart of NVIDIA's announcements is Project GR00T, NVIDIA's initiative focused on humanoid robot development. The project introduces six specialized workflows covering everything from environment generation to robot control and perception. These tools help developers tackle complex challenges like teaching robots to walk, manipulate objects, and interact naturally with their surroundings.
Hugging Face’s LeRobot framework, designed to extend the paradigms of popular AI libraries like Transformers and Diffusers into robotics, will integrate with NVIDIA Isaac Lab and Jetson platforms. This will enable developers to share and utilize models and datasets to improve robot training, policy development, and deployment.
Unlike language models that can learn from internet text, robots need physical interaction data that's harder to collect at scale. NVIDIA's Isaac Lab framework addresses this by enabling developers to train robots in simulation before deploying them in the real world.
NVIDIA also introduced new tools to help developers build world models—AI representations of environments that predict responses to a robot's actions. The Cosmos tokenizer processes visual data up to 12 times faster than current solutions, while maintaining high image quality. The NeMo Curator tool speeds up video processing by seven times, making it easier for developers to train robots using real-world data.
Leading robotics companies including Boston Dynamics, Agility Robotics, and 1X Technologies are already adopting these new tools. Eric Jang, vice president of AI at 1X Technologies, highlighted the benefits: "NVIDIA Cosmos tokenizer achieves really high temporal and spatial compression of our data while still retaining visual fidelity."