Nvidia neural rendering deep dive — Full details on DLSS 4, Reflex 2, mega geometry, and more

During CES 2025, Nvidia held an Editors' Day where it provided technical briefings on a host of technologies related to its upcoming RTX 50-series GPUs. It was an all-day event, kicked off with the Neural Rendering session that covered topics including DLSS 4, Multi Frame Generation, Neural Materials, Mega Geometry, and more.

AI and related technologies will be front and center on the RTX 50-series, though some of the upgrades will also work on existing RTX GPUs, going all the way back to the original 20-series Turing architecture. We also have separate articles on the Blackwell architecture, RTX 50-series Founders Edition cards, RTX AI and generative Ai for games, Blackwell for professionals and creators, and how to benchmark Blackwell and MFG. There's a ton of information to cover, and we have the full slide decks for each session.

Due to time constraints — Intel's Arc B570 launches tomorrow, and there's still all the testing to get done for the RTX 5090 and 5080 launches later this month — we won't be able to cover every aspect of each discussion, but let us know in the comments any questions you might have, and we'll do our best to respond and update these articles with additional insight in the coming days.

Image 1 of 22

The Neural Rendering session was full of interesting information and demos, most of which should be made publicly available. We'll add links to those if and when they go live. At a high level, many of the enhancements stem from the upgraded tensor cores in the Blackwell architecture. The biggest change is native support for the FP4 (4-bit floating-point) data type, with throughput that's twice as high as the prior generation's FP8 compute. Nvidia has been training and updating its AI models to leverage the enhanced cores, and it lumps these all under the Blackwell Neural Shaders header.

Nvidia lists five new features: Neural Textures, Neural Materials, Neural Volumes, Neural Radiance Fields, and Neural Radiance Cache. Read through the list fast, and you might start to feel a bit neurotic...

We already discussed some of the implications of the Neural Materials during Jensen's CES keynote, where future games could potentially reduce memory use for textures and materials by around one-third. That will require game developers to implement the feature, however, so it won't magically make any 8GB GPUs — including the RTX 5070 laptop GPU and most likely an unannounced RTX 5060 desktop GPU — work better. But for games that leverage the feature? They could look better while avoiding memory limitations.

Nvidia is working with Microsoft on a new shader model called Cooperative Vectors that will allow the intermingling of tensor and shader code, aka neural shaders. While this should be an open standard, it's not clear if there will be any support for other GPUs in the near term, as this very much sounds like an RTX 50-series-specific enhancement. It's possible to use these new features on other RTX GPUs as well, though our understanding is that the resulting code will be slower as the older architectures don't natively support certain hardware features.

Nvidia also showed off some updated Half-Life 2 RTX Remix sequences, with one enhancement called RTX Skin. With existing rendering techniques, most polygons end up opaque. RTX Skin uses AI training to help simulate how light might illuminate certain translucent objects — like the above crab. There was a real-time demo shown, which looked more impressive than the still image, but we don't have a public link to that (yet).

Another new hardware feature for the RTX 50-series is called linear-swept spheres. While the name is a bit of a mouthful, its purpose is to better enable the rendering of ray-traced hair. Modeling hair with polygons can be extremely demanding — each segment of a hair would require six triangles, for example, and with potentially thousands of strands, each having multiple segments, it becomes unwieldy. Linear-swept spheres can trim that to two spheres per segment, requiring 1/3 as much data storage.

The number of polygons in games has increased substantially over the past 30 years. High-end games in the 1990s might have used 1,000 to 10,000 polygons per scene. Recent games like Cyberpunk 2077 can have up to 50 million polygons in a scene, and with Blackwell and Mega Geometry, Nvidia sees that increasing another order of magnitude.

Interestingly, Nvidia mentions Unreal Engine 5's Nanite technology that does cluster-based geometry as a specific use case for Mega Geometry. It also specifically calls out BVH (Bounding Volume Hierarchy), though, so the may be specifically for ray tracing use cases.

Nvidia showed a live tech demo called Zorah, both during the keynote and during this session. It was visually impressive, making use of many of these new neural rendering techniques. However, it's also a tech demo rather than an upcoming game. We suspect future games that want to utilize these features may need to incorporate them from an early stage, so it could be a while before games ship with most of this tech.

YouTube

Watch On

Image 1 of 30

The second half of the Neural Rendering session was more imminent, with VP of Applied Deep Learning Research Bryan Catanzaro leading the discussion. Succinctly, it was all oriented around DLSS 4 and related technologies. More than anything else, DLSS 4 — like the RTX 40-series' DLSS 3 — will be key to unlocking higher levels of "performance." And we put that in quotes because, despite the marketing hype, AI-generated frames aren't going to be entirely the same as normally rendered frames. It's frame smoothing, times two.

Nvidia notes that over 80% of gamers with an RTX graphics card are using DLSS, and it's now supported in over 540 games and applications. As an AI technology, continuing traning helps to improve both quality and performance over time. The earlier DLSS versions used Convolutional Neural Networks (CNNs) for their models, but now Nvidia is bringing a transformers-based model to DLSS. Transformers have been at the heart of many of the AI advancements of the past two years, allowing for far more detailed use cases like AI image generation, text generation, etc.

With the DLSS transformer model, Nvidia has twice the number of parameters and it uses four times the compute, resulting in greatly improved image quality. Moreover, gamers will be able to use the Nvidia App to override the DLSS modes on at least 75 games with the RTX 50-series launches, allowing the use of the latest models.

Image 1 of 2

Above are two comparison shots for Alan Wake 2 and Horizon Forbidden West showcasing the DLSS transformers upgrades. Alan Wake 2 uses full ray tracing with ray reconstruction, which apparently runs faster in addition to looking better than the previous CNN model. Forbidden West was used to show off the improved upscaling quality, with a greatly increased level of fine detail.

We haven't had a chance to really dig into DLSS transformers yet, but based on the early videos and screenshots, after years of hype about DLSS delivering "better than native" quality, it could finally end up being true. Results will likely vary by game, but this is one of the standout upgrades — and it will be available to all RTX GPU owners.

The DLSS transformer models do require more computing, and so may run slower, particularly on older RTX GPUs, but it's possible we could see DLSS Performance mode upscaling (4X) with the new models that look better and perform better than the old DLSS CNN Quality mode (2X upscaling).

Image 1 of 3

We're less enthusiastic about Multi Frame Generation (MFG), if only because of the amount of marketing hype behind it. With the new hardware flip metering to help smooth things out, coupled with faster frame generation algorithms, on the one hand MFG makes sense. Instead of interpolating one frame, you generate three. Poof! Three times the performance uplift!

But on the other hand, that means one-fourth the sampling rate. So if you're running a game with MFG and getting 240 FPS, that's really rendering at 60 FPS and quadrupling that with MFG. And that specific example would probably look and run great. What will be more problematic is games and hardware that can't get 240 FPS... for example, an RTX 5070 might only run at 120 FPS. What would that feel like?

Based on what we've experienced with DLSS 3 Frame Generation, there's a minimum performance level that you need to hit for things to feel "okay," with a higher threshold required for games to really feel "smooth." With single frame generation, we generally felt like we needed to hit 80 FPS or more — so a base framerate of 40. With MFG, if that same pattern holds, we'll need to see 160 FPS or more to get the same 40 FPS user sampling rate.

More critically, framegen running at 50 FPS, for example, tended to feel sluggish because user input only happened at 25 FPS. With MFG, that same experience will now happen if you fall below 100 FPS.

Of course, you won't see perfect scaling with framegen and MFG. Regular DLSS 3 framegen would typically give you about a 50% increase in FPS — but Nvidia often obfuscates this by comparing DLSS plus framegen with pure native rendering.

If you were running at 40 FPS native, you'd often end up with 60 FPS after framegen — and a reduced input rate of 30 FPS. What sort of boost will we see when going from single framegen to 4X MFG? That might give us closer to a direct doubling, which would be good, but we'll have to wait and see how it plays and feels in practice.

Nvidia's own examples in the above slide suggest the performance improvement could range from as little as 33% (Alan Wake 2) to as much as 144% (Hogwarts Legacy) — but that's using an RTX 5090. Again, we'll need to wait and see how this works and feels on lesser 50-series GPUs.

Image 1 of 6

Wrapping things up, the other big new item will be Reflex 2. It will eventually be available for all RTX GPUs, but it will first be available with the RTX 50-series. Where Reflex delayed input sampling until later in the rendering pipeline, Reflex 2 makes some interesting changes.

First, our understanding is that, based on past and current inputs, it will project where the camera will be before rendering. So if, as an example, your view has shifted up two degrees over the past two input samples, based on timing, it could predict the camera will actually end up being shifted up three degrees. Now here's where things get wild.

After all the rendering has finished, Reflex 2 will then sample user input once more to get the latest data, and then it will warp the frame to be even more accurate. It's a bit like the Asynchronous Space Warp (ASW) used with VR headsets. But the warping here could cause disocclusion — things that weren't visible are now visible. Reflex 2 addresses this with a fast AI in-painting algorithm.

The above gallery shows a sample of white pixels that are "missing." At the Editors' Day presentation, we were shown a live demo where the in-painting could be toggled on and off in real-time. While the end result will depend on the actual framerate you're seeing in a game, Reflex and Reflex 2 are at their best with higher FPS. We suspect the warping and in-painting might be more prone to artifacts if you're only running at 30 FPS, as an example, while it will be less of a problem in games running at 200+ FPS. Not surprisingly, Reflex 2 will be coming first to fast-paced competitive shooters like The Finals and Valorant.

And that's it for the Neural Rendering session. The full slide deck is available above, and we glossed over some aspects. A lot is going on with AI, needless to say, and we're particularly curious to see what happens with things like Neural Textures. Could we see that applied to a wider set of games?

We asked Nvidia about that, and it sounds like some game studios might be able to "opt-in" to have their assets recompressed at run-time, but it will be on a game-by-game basis. Frankly, even if there's a slight loss in overall fidelity, if it could take the VRAM requirements of some games from 12–16 GB and reduce that to 4~6 GB, that could grant a new lease on life to 8GB GPUs. But let's not count those chickens until they actually hatch.