Nvidia’s Project Holodeck Brings Photorealistic Graphics To Shared VR Environments

Nvidia revealed Project Holodeck during CEO Jen-Hsun Haung's opening keynote at the company's GPU Technology Conference (GTC), signaling that photorealistic graphics in shared VR environments is not such a distant goal. Presumably the demonstration, featuring a $1.9 million automobile, ran on the company's newest Volta GV100 GPUs, announced a few moments later during the presentation.

Nvidia’s Holodeck lets you visualize and explore virtual representations of real objects in ways not possible in the real world.

It's always difficult to parse a technology showcase from a product, but it appears that Project Holodeck is intended for "creators" (Nvidia's word)—design engineers and those with whom they collaborate. It uses a version of Epic's Unreal Engine 4, as well as Nvidia's own GameWorks, VRWorks, and DesignWorks. It's an experimental multi-user virtual environment with real-time photorealistic graphics capabilities and true-to-life physics.

All of that is to say that today's virtual reality experiences can be incredibly compelling, and the sense of immersion that spatial tracking and motion controllers provide can fool your brain into believing that you’re inside a digital environment, even if the graphics quality of the experience is somewhat compromised, but the blending of the real world and digital environments in VR is yet another frontier.

Nvidia's live demonstration of the Holodeck Project had Huang calling upon Christian Koenigsegg—not exactly a household name, but kind of a big deal in the automotive industry. Koenigsegg Automotive AB makes some of the world’s most expensive, exotic sports cars. Koenigsegg joined the presentation via VR to show off his company’s latest creation, the Koenigsegg Regera hypercar.

The demonstration started with four high-fidelity robot avatars floating in a large three-dimensional grid space devoid of anything else at all. Then, Huang asked to see the Koenigsegg Regera, and suddenly a wireframe silhouette of the latest Koenigsegg hypercar faded into view, followed immediately by the grid space environment transitioning to an infinite white space and the car coming into full view in photorealistic rendering. The effect was exactly like the scene in The Matrix where Neo experiences the Construct for the first time.

Nvidia’s Project Holodeck demo isn’t just some passive experience that lets you view objects in photorealistic renderings. The software also lets you interact with the objects in front of you. Koenigegg reached out and opened the Regera’s driver-side door and proceeded to step into the car to see the interior details.

The laws of physics apply in Project Holodeck, and this allows the virtual objects to react to your touch as if they were real. Koenigsegg grabbed the virtual steering wheel to show how Nvidia leverages VR PhysX technology so that your avatar’s hands don’t pass through virtual objects.

The demonstration didn’t end there. Seeing a photorealistic representation of the car that you can interact with is all well and good, but it doesn’t beat seeing the real thing in person. Project Holodeck lets you see the inner details of the car in ways that aren’t possible in real life.

The model of the Koenigsegg Regera that Nvidia used in the Project Holodeck demo was painstakingly rendered with every detail you can imagine (50 million polygons, according to Nvidia). And because it’s a digital rendering, you can do things like add a transparency filter to the object that lets you see under the shell of the car and into all of its mechanical bits.

Project Holodeck also lets you see an exploded view of the car. When the demonstration switched to the exploded view, the car erupted into a cluster of disassembled parts in an animation that played out in real-time.

Nvidia didn’t say when we could expect to see Project Holodeck experiences reach the public, but the company offers the opportunity to follow the project.