Developing AI for humanoid robots involves tackling many open research challenges – in safety, dexterity, visual understanding, and much more. It helps to compare notes with other labs tackling similar challenges, in order to accelerate progress towards a future of NEOs doing all the tasks needed to keep your home in order autonomously.
To that end, 1X AI and NVIDIA are pleased to announce our research collaboration effort. As a first step, the teams worked together to prepare an autonomy demo for Jensen Huang’s GTC 2025 Keynote, featuring NEO doing a dish loading task autonomously.
The following is a look into where, how and when we taught NEO to do the dishes with the NVIDIA team.
To make this collaboration possible, the 1X AI Team created a dataset API for NVIDIA to access data collected from 1X offices and employee homes, and an inference SDK to serve model predictions at a continuous 5Hz vision-action loop using an onboard NVIDIA GPU in NEO’s head or an offboard GPU.
A crucial step when onboarding a new learning codebase onto NEO is to verify correctness, i.e., overfitting a baseline model to a small amount of demonstration data and making sure that the time synchronization between images and actions is consistent all the way from data collection to training to runtime inference.
We demonstrate this by working with the GEAR team to train a single end-to-end neural network based on the NVIDIA GR00T N1 model to autonomously grasp a cup, hand it over to the other hand, and place it in a dishwasher to showcase how NEO fits compactly into the kitchen space while still having the kinematic reach to carry the cup from sink to dishwasher.
This is a good “first task” to learn because it checks for basic compatibility of an external research codebase with the logging and inference architecture. The obvious next step after verifying correctness is to feed thousands of hours of internally collected NEO data into the model.
Over the course of a week, our teams developed this model at a 1X employee’s home, swapped notes on action spaces, control frequencies, and other imitation learning tricks needed to get good performance on NEO Gamma. Moments like these – where friends are just hanging out in the home while a NEO does dishes in the background – will soon become an everyday occurrence.
When working in homes, the safety of NEO Gamma becomes particularly evident. NEO’s mechanically compliant and safe design allowed engineers to get in extremely close quarters with the robot while testing a variety of experimental architectures.
Looking Forward Our teams are both looking forward to continuing to learn from each other and push the industry forward. We hope that together we can accelerate our path to humanoids living and learning among us and providing a helping hand wherever one is needed.