Cooking With NEO Beta and Nick DiGiovanni
A behind the scenes look into the production of Nick DiGiovanni and NEO Beta's cooking challenge including BTS content and technical details.
A behind the scenes look into the production of Nick DiGiovanni and NEO Beta's cooking challenge including BTS content and technical details.
1X's mission is to provide an abundant supply of physical labor via safe, intelligent androids. Our environments are designed for humans, so we design our hardware to take after the human form for maximum generality. To make the best use of this general-purpose hardware, we also pursue the maximally general approach to autonomy: learning motor behaviors end-to-end from vision using neural networks.
We deployed this system on EVE for patrolling tasks in 2023, and are now excited to share some of the new capabilities our androids have learned purely end-to-end from data:
Every behavior you see in the above video is controlled by a single vision-based neural network that emits actions at 10Hz. The neural network consumes images and emits actions to control the driving, the arms, gripper, torso, and head. The video contains no teleoperation, no computer graphics, no cuts, no video speedups, no scripted trajectory playback. It's all controlled via neural networks, all autonomous, all 1X speed.
To train the ML models that generate these behaviors, we have assembled a high-quality, diverse dataset of demonstrations across 30 EVE robots. We use that data to train a “base model” that understands a broad set of physical behaviors, from cleaning to tidying homes to picking up objects to interacting socially with humans and other robots. We then fine-tuned that model into a more specific family of capabilities (e.g. a model for general door manipulation and another for warehouse tasks) and then fine-tuned those models further to align the behavior with solving specific tasks (e.g. open this specific door). This strategy allows us to onboard new skills in just a few minutes of data collection and training on a desktop GPU.
All of the capabilities shown in the video were trained by our android operators. They represent a new generation of "Software 2.0 Engineers'' who express robot capabilities through data instead of writing code. Our ability to teach our robots short mobile manipulation skills is no longer constrained by the number of AI engineers, so this creates a lot of flexibility in what our androids can do for our customers.
If you find this work interesting, we’d like to call attention to two roles that we are hiring for to accelerate our mission toward general-purpose physically embodied intelligence:
Over the last year we’ve built out a data engine for solving general-purpose mobile manipulation tasks in a completely end-to-end manner. We’ve convinced ourselves that it works, so now we're hiring AI researchers in the SF Bay Area to scale it up to 10x as many robots and teleoperators. We're looking for experts in imitation learning, reinforcement learning, large-scale training, and skills relevant to scaling up deployments of autonomous vehicles. You'll be working in a fast-paced team of generalists that ship features to our fleet on a 24-hour release cycle. The work is a mix of pioneering new learning algorithms and fixing speed bottlenecks in our data flywheel. We are relentless in simplifying algorithms and infrastructure as much as possible.
We're also hiring android operators in both our Oslo and Mountain View offices to collect data, train models with that data, and evaluate those models. Unlike most data collection jobs, our teleoperators are empowered to train their own models to automate their own tasks and think deeply about how data maps to learned robot behavior. If you want to experience what it is like to live in a real-life "Westworld", we'd love for you to apply.
We also have other open roles across mechanical, electrical, and software disciplines that make the foundation possible to ship all of this cutting-edge ML technology. Follow 1x_tech on X for more updates, and join us in living in the future.
1-minute summary
In the ever-evolving landscape of robotics, 1X is leading the charge towards redefining the capabilities of humanoids for real-world impact. Unlike traditional systems that excelled in controlled lab environments but stumbled when faced with the unpredictability of the real world, our goal is to design general purpose robots that will function effectively in any scenario.
At the heart of this endeavor is the development of a two-legged robot capable of safe and dynamically dexterous tasks. NEO is 1.65 meters tall, weighs 30 kilograms, has 55 degrees of freedom (including hands), and is made of meticulously crafted components.
January 2024, 1X will be hosting “Open House”. Allowing a selected group of people to learn more about how we’re working to achieve our goals and get a taste of what it means to be a vertically integrated robotics company.
The journey towards creating dynamic and safe humanoids has its challenges. One significant problem is the lack of commercially available motors and transmission systems that allow the design of passively safe and high-performance robots.
For robots to function alongside humans, safety is paramount. Robots built with highly geared and/or low efficiency transmission elements (harmonic drives, leadscrews…) show very little compliance, making the robot interactions with the world very rigid and unsafe. In addition to that, the high friction, high weight and high inertia of such systems end up translating into slow, clumsy, or unreliable robots.
Moreover, most of the off-the-shelf motors are made for applications where the torque-to-weight ratio - or torque-to-volume ratio - is very low. Therefore, by using such motors, it is often necessary to use a high gear ratio transmission to achieve the desired torque at the output. This high gear ratio acts as a multiplier of not only the torque, but also the inertia and friction. This is what we call internally a “non-transparent transmission”.
In robotics, especially when building humanoid robots, it is crucial to have a profound influence over the form factor and functionality of all subsystems and components. Almost every inch of the 1X hardware and software is designed in-house, allowing us to fine-tune every aspect of our robots keeping safety, performance, and cost under our control when the humanoid hits the market.
1X’s vertical integration approach expands wide and deep into our technology stack. One of the main layers of this stack is what we call "transparent transmission". In simple terms, our cable drive transmissions are designed so that we can precisely determine the torque at the output (joint) based on measuring the torque at the input (motor), without needing expensive torque sensors. The low friction and low inertia along the transmission is negligible, ensuring its “transparency”.
Another important asset of this “transparent transmission” is our in-house developed electric motors. We’ve designed them in a way that the torque-to-weight ratio (and torque-to-volume ratio) is very high. This allows us to package high torque joints without the need of a high gear ratio transmission, making the robot joints very low in terms of inertia, mechanical losses, and weight. This “quasi-direct” concept takes very little space, allowing NEO to have a form factor similar to a 1.65-meter human.
All above help us to build humanoid robots that are intrinsically compliant, safe, low cost, and dynamic allowing NEO to carry or lift human-like payloads.
The open house in January 2024 serves as an opportunity for robotics engineers to witness firsthand the strides we’re making to have robot companions in our daily lives. Our CEO, Bernt Øivind Børnich, and top software and hardware engineers will share insights into our journey of crafting our humanoids.
Submit your interest in participating here.
In 2019, Sunnaas Hospital in Norway started a project to use a robotic assistant named EVE in healthcare. This project, called HIRO (Human Interactive Robotics for Healthcare), tested how Androids can assist healthcare workers every day.
HIRO is a project supported by the Research Council that includes Sunnaas Hospital, 1X developers, the Institute for Energy Technology (IFE), and the Skien municipality. They worked together to make a helpful Android that can do everyday tasks like moving equipment, fetching things, and helping patients get around. The main purpose was to create a solution for the expected shortage of healthcare workers in the future.
Sunnaas Hospital tested the robot EVE with healthcare workers to see how well it works in real treatment situations. They compared how things went with EVE's help and without it. Before these tests, they asked healthcare workers for their opinions, needs, and thoughts. They found that when EVE did simple, non-medical tasks, the human staff had more time to focus on patient care using their skills and knowledge.
The project is about more than just checking if the robot assistant works well. It's also about understanding how robots and people can work together in healthcare. The goal is to learn what healthcare workers think about having a robotic helper and what tasks EVE can do best.
In essence, the robotic assistant is conceived as a facilitator, enhancing efficiency in healthcare without diminishing the essential human touch.
The work being done at Sunnaas Hospital is a careful assessment of how Androids can be usefully and trustfully deployed in healthcare.
Every day our Androids train for practical tasks and smarter behavior in The Studio. Observe the ongoing innovation behind the scenes at 1X.
At The Studio in Moss, Norway, our Androids train for practical tasks, general use and industry-specific behavior using Embodied Learning. Inspired by advancements in digital assistants and self-driving cars, 1X’s method for data collection diverges from traditional coding and predetermined algorithms. Using VR Teleop, 1X Operators guide the Androids through diverse real-world scenarios, offering an intuitive understanding of the task’s difficulty and feasibility. When data is collected at scale, the Android has learned a new skill.
It's less about set algorithms and more about practical learning for navigating the world that was built to fit you and me, or ‘the average human shape’. Embodied Learning is a hands-on method for data collection that allows the Androids to adapt and navigate diverse situations in the real world, contributing to their ability to perform tasks autonomously in dynamic and unstructured environments.
Follow 1X and The Studio on YouTube and Instagram, to see how we put our tech to the test, as we expand the skills of our droids.
Engage with us as we collectively steer the evolution of Android autonomy.
In a recent episode of the TomorrowTalk podcast, Eric Jang, VP of AI at 1X, discusses the evolving landscape of artificial intelligence and humanoid robotics with host Sabrina Helper.
The conversation spans topics ranging from academia versus startup dynamics in AI, critiques of effective altruism and AI safety, to the intersection of AI and politics. Jang shares his thoughts on AI companionship, the strategic use of AI chatbots, and addresses critical questions on achieving Artificial General Intelligence.
Jang highlights the societal impact of AI and touches on 1X's approach to managing workforce disruption responsibly. Envisioning a post-AGI future, he reflects on the evolving dynamics between humans and robots.
The podcast provides a concise yet comprehensive exploration of the future possibilities and challenges in the realm of AI and humanoid robotics.
Find the full podcast below:
The Industrial Revolution was a pivotal moment in human history, reshaping our lives by dramatically increasing available energy. It replaced reliance on human and animal muscle power with fossil fuels, setting the stage for innovations like the steam engine and the internal combustion engine. These inventions transformed agriculture and transportation, feeding more people and enabling faster and more extensive goods transport.
Fast forward to the present day, and we are once again on the brink of another transformative change. This time, the driving force is not energy but computation, specifically in the form of artificial intelligence (AI). However, similar to the Industrial Revolution, the full potential of this change won't be realized until we can deploy this computational power into the world. Enter the age of androids – the new engines of AI.
Today, we have enormous data centers processing vast amounts of information, all interconnected through the global internet. These data centers are the brains behind the digital revolution, but they lack a means to interact with the physical world in a versatile way. While we have made great strides in creating smart homes, smart cars, and smart factories, these are just the tip of the iceberg. Androids represent a broader platform for the computational power of our civilization to have a tangible impact on the world.
At its core, our world is designed by and for humans, which makes the human form the most effective means of interfacing with it. Androids are humanoid robots, designed to mimic the human body, with legs, arms, and hands that provide them with similar reach and strength. These robots are rapidly advancing in their ability to understand human language and interact deftly with the world, bringing us closer to a seamless integration of AI into our daily lives.
Two hundred years ago, it would have been challenging to imagine the towering skyscrapers, jumbo jets, electric lights, and other innovations that emerged from the shift from muscle power to fossil fuels. In much the same way, it is difficult today to fully grasp the profound impact that a widespread proliferation of androids will have on our world. However, one thing is clear: the advent of androids will be equally transformative.
The potential benefits of widespread android deployment are vast. With their ability to assist in various tasks and augment human capabilities, androids have the power to raise the living standards of people across the globe. They can enhance productivity in agriculture, construction, healthcare, and countless other industries. Androids can take on dangerous and repetitive jobs, allowing humans to focus on creative and strategic endeavors. They can revolutionize education, providing personalized and immersive learning experiences. Moreover, they can foster a more inclusive and equitable society, ensuring that the benefits of AI and automation are accessible to all.