Nvidia Reveals DLSS 3.5: AI-Powered Ray Reconstruction

Nvidia DLSS — Deep Learning Super Sampling — has come a long way over the past five years since it was first introduced. While the deep learning aspect of the name remains, the latest iterations have focused on other techniques besides super sampling. At Gamescom 2023, Nvidia revealed DLSS 3.5 with AI-powered ray reconstruction, the newest feature under the DLSS umbrella, and it's available on all RTX GPUs, unlike DLSS 3's Frame Generation.

The Nvidia RTX GPUs are some of the best graphics cards, but if there's one thing you can count on, it's that tomorrow's games and graphics technologies will be even more demanding than current offerings. Leveraging the power of AI to reduce some of the rendering work can potentially extend the life of older GPUs. That's assuming that future games implement the new techniques, naturally.

YouTube

Watch On

Image 1 of 8

Ray tracing remains one of the most demanding graphics rendering techniques, some would even call it the holy grail of graphics rendering. Calculating the final pixel colors based on reflections, refractions, and the physical properties of various materials takes a lot of computational power. Rather than doing ray tracing calculations on every pixel on the screen — with multiple rays per pixel — DLSS 3.5 uses AI to fill in the gaps. Nvidia calls this "ray reconstruction."

The same process can be done in other ways, via denoisers — algorithms designs to interpolate between ray traced pixels. This can be both temporal accumulation (pixels over multiple frames) or spatial interpolation (blending between neighboring pixels). In professional 3D rendering applications, you'll often see multiple passes where the initial results are a fast and approximate rendering, then as more rays are cast and the results improve, the speckled nature of the resulting view improves. DLSS 3.5 can be used in both games and applications to similar effect.

Most games with ray tracing effects already use denoisers to improve the overall quality, but just as image upscaling has benefited from deep learning, denoising can similarly benefit. Nvidia says that Ray Reconstruction has been trained to recognize different ray-traced effects, and it also leverages both temporal and spatial pixels to deliver the best result.

To demonstrate the potential of DLSS 3.5, Nvidia provided several sample slides showing the current rendering compared with DLSS 3.5's Ray Reconstruction.

Image 1 of 6

Most of the DLSS 3.5 versus "standard" images should be pretty obvious about what Ray Reconstruction brings to bear, but in Portal RTX, the enhanced graphics show much smoother gradients and blending with less splotchiness.

For Cyberpunk 2077, running in RT Overdrive, there are several improvements. First, there's again an improvement in quality with less splotchiness. There are also clearer reflections, for example on the top of the vehicle, or in the puddles on the street. Without Ray Reconstruction, the (upside-down) text in the puddle ends up smeared and illegible, while with DLSS 3.5 you can make out the letters. Finally, the headlights of the car in the first image light up more distant objects like the railing, which appears to be a change facilitated by having fewer, higher quality rays plus Ray Reconstruction.

Nvidia also provided a rough performance comparison in Cyberpunk 2077 between native (DLSS off), DLSS 2 upscaled (Super Resolution), DLSS 3 (SR plus Frame Generation), and DLSS 3.5 (SR + FG + Ray Reconstruction). We were told that Cyberpunk 2077 has relatively complex denoisers normally, and DLSS 3.5 is able to replace those, resulting in the performance increase along with improved lighting quality.

Finally, as a non-gaming example of DLSS 3.5, Nvidia showed D5 Render running in full ray tracing preview mode, with and without Ray Reconstruction. On the left, the preview looks speckled and splotchy — this would improve in quality the longer you leave the camera unchanged. DLSS 3.5 meanwhile provides a much better quality result much faster.

Again, it's important to emphasize that Ray Reconstruction and DLSS 3.5 will be available on all RTX graphics cards, from the original 20-series, through the previous 30-series, and on to the current 40-series and future offerings. If you have an Nvidia RTX GPU and you run a game that supports DLSS 3.5, you can benefit from the feature.

At the same time, let's also be clear that this only benefits games that use ray tracing effects, and it also requires the games to adopt DLSS 3.5. And while DLSS 3.5 might work on all Nvidia RTX GPUs, it will provide zero benefit for AMD and Intel GPU owners. That's going to limit its appeal somewhat, but for games that are already adding DLSS 3 support — which only runs on RTX 40-series cards — this seems like it would potentially be at least as interesting as Frame Generation.

DLSS 3.5 will come out this fall, and support has already been announced for Cyberpunk 2077: Phantom Liberty, Portal with RTX, and Alan Wake 2. Yeah, that's not a lot of games right now, but we expect more will be announced in the coming months. Support will also be present in Nvidia's Omniverse platform, Chaos Vantage, and D5 Render.

Image 1 of 35

Nvidia also had other items of interest, though by far the most exciting will be the announcement of a community mod using RTX Remix for Half-Life 2. Don't get too excited, though, as work is only just starting. It does have support from Nvidia, though, which means the mod should eventually get finished and released rather than ending up as vaporware.

Elsewhere, Nvidia was pushing GeForce NOW Ultimate, with RTX 4080 (equivalent) SuperPods now deployed across the U.S. and Europe and supporting up to 240 fps streaming — for those who have a 240 Hz monitor but not a powerful GPU to go with it. Xbox Game Pass support is also coming to GeForce NOW, starting on August 24.

Nvidia also showed off some updates to its Avatar Cloud Engine, now with SteerLM — a language model that allows more customization in the types of responses you might get from an AI. This could be used with future games to allow NPCs to have real-time responses to human speech, or perhaps for more interesting (but still just as likely to be... questionable) ChatGPT-type sessions.