The CES consumer electronics show in Las Vegas, which has become the show to highlight the latest and greatest in consumer technology, is about to get in full swing. Nvidia Corp. has been a mainstay at the show for years, first through its gaming business, but over the years it has broadened its presence into areas such as autonomous vehicles and artificial intelligence.
One of the highlights from its crop of announcements at CES 2023 is the advancements it’s making in the field of robotics. Today Nvidia unveiled the newest release of its Isaac Sim simulation application for developing and testing artificial intelligence robots. Using Isaac Sim, developers and researchers can train AI robots for various tasks and use cases across different industries.
Isaac Sim is an app that runs on Nvidia’s core platform, Omniverse, which enables photorealistic 3D simulations and digital twins. Omniverse powers several other Nvidia apps, such as DRIVE Sim for autonomous vehicle development. Isaac, however, is designed specifically for robotics. The Isaac toolkit includes several AI-based robot development components.
Training robots in a virtual environment is much faster and more accurate than with physical robots. When training an actual robot, the testers would need to create a wide range of scenarios such as walking up and down stairs and traversing gravel roads, potholes, rough terrain and slippery surfaces, among much more. This is a long, expensive process and it can take years to fully train a robot.
With Isaac Sim, the training can be done with a digital twin, the data uploaded to the physical robot, which is then fully trained. When Nvidia opened its Endeavor building, it used a previous version of Isaac to train a robot to move about the facility. When the building opened, the robot was fully functional and able to navigate the environment.
It’s important to understand that the use of robots is no longer the stuff of science fiction but is something that has become common across several industries. Robots are now being used for last-mile delivery, pick-and-place, agriculture, industrial robots, cameras, sensors and more. Recently, I interviewed the head of technology for the Dubai Police Department, which has robots in the field to provide information to citizens but also to augment the police department because it can provide additional “eyes.”
One component in Isaac is SimReady assets, a collection of 3D assets for building industrial spaces like warehouses and manufacturing environments. Then there’s the Replicator, a set of tools for collecting and labeling robotics data. Isaac Robot Operating System is a collection of software packages that are hardware–accelerated and targeted at the ROS development community. CuOpt is the newest addition to Isaac, providing a route optimization and planning for managing fleets of robots. Isaac also includes various pre-trained models based on Nvidia AI, optimized for specific environments.
“The star of the show is Isaac Sim,” said Gerard Andrews, senior product marketing manager of robotics at Nvidia. “This complex software is going to drive autonomous robots. Isaac Sim is a flexible platform that enables various robotic apps, which companies are already deploying.”
Isaac Sim launched in 2021 in open beta, allowing developers to test the app in their own environments for free. Nvidia has since made Isaac Sim available on Omniverse Cloud and from leading cloud services, so developers can design apps from anywhere. The company continues to invest in robotics, with many types of robots now leveraging its technology — from autonomous mobile robots to last–mile delivery robots to agricultural robots. More than a million developers have used at least one part of Isaac, according to Andrews.
“Isaac Sim use cases continue to roll in.,” he said. “Even before this new release, we made a lot of progress across a variety of robotic applications.”
In the latest Isaac Sim release, Nvidia introduced tools specifically for warehousing and logistics. Using the tools, developers can create virtual robotic worlds that mimic real facilities or even test processes on factory floors that may be dangerous or expensive. This release of Isaac Sim comes with a conveyor belt utility for testing conveyor belts, as well as integration with cuOpt.
Another new feature is a people simulation, where developers can drop characters into an environment, assign tasks and have characters perform the tasks. The simulation is useful not only in imitating human behavior but also in testing scenarios that are difficult to recreate or capture in real life.
Furthermore, Nvidia has improved sensor performance in Isaac Sim, with RTX lidar support and added lidar models such as Ouster, Slamtec and Hesai. It also added ROS/ROS2 replicator, for synthetic data, with post-processing support for more robust datasets. Andrews said these are all the features customers have been asking for.
“In every subsequent release, we’re striving to add tools to make the simulations more realistic and increase the use cases,” he said. “The more apps you can have in one simulator, the more valuable it will be to the broader ecosystem. We believe simulation is going to become critical in advancing robotics.”
The era of robots has arrived, and it will have a significant impact on society because they can perform dangerous tasks or redundant operations that people are not good at. They can aid disabled people and keep us safe. Training robots has never been easy, but Nvidia has simplified the process by providing a full turnkey set of tools to do so.
