AI Meets VR When Nvidia Trains Isaac On Holodeck

Earlier this year, Nvidia revealed an ambitious VR collaboration tool, called Holodeck, that will offer photo-realistic graphics and physically rendered environments. The company also introduced Isaac, a tool for training robots in a controlled digital environment. Now, the company has revealed that a combination of both tools can help researchers safely train robots that interact with human beings.

Nvidia’s Holodeck platform is still an experimental vehicle that lets a team of people view real-world objects rendered in photorealistic VR. For example, it lets you load CAD drawings into virtual environments and display them in incredible detail. What's more, Holodeck can simulate real-world physics so that virtual objects react to other objects, people, and the environment as they would in the real world.

When Nvidia revealed Holodeck during its GTC gathering in May, the company showed a live demonstration of its potential, showing off the latest Koenigsegg car in an impressive live demo highlighting the car’s features and components in an exploded view of the car's CAD files. It was during the same GTC event that Nvidia revealed its Isaac deep learning AI training platform.

But here, at Siggraph, Nvidia demonstrated that AI researchers can take advantage of Holodeck’s true-to-life physics to help reduce the cost and risk of training robots that interact with humans, while still being able to do practical experiments of robots interacting with humans. Holodeck offers a perfect environment for AI experimentation because it removes much of the risk from the equation.

Isaac on Holodeck lets developers load new algorithms into the robot and then hop into the VR environment to test its implementation. Thanks to Holodeck’s accurate physics and photorealistic visuals, AI researchers can experience with what it would be like to interact with the real robot. Following a successful test, researchers would be able to export the “AI brain” from Isaac and embed it into the real-world robot.

Nvidia is showcasing Isaac on Holodeck at the Siggraph conference. If you’re attending the show, you can put on a VR headset and sit down with Isaac to play a game of dominoes. Nvidia is also showcasing the Holodeck Koenigsegg demo from GTC.