Nvidia's Deep Learning Super Sampling Technology, Explored
Nvidia's new DLSS (Deep Learning Super Sampling) technology is one of the Turing architecture's most promising, as we first showed in our GeForce RTX 2080 Ti Founders Edition review. But it is also the most mysterious. The company isn't going into depth on how DLSS works. Of course, we wanted to know more. So, after hours of testing and image analysis, we think we have the answer.
DLSS, According to Nvidia
In its descriptions of DLSS' inner workings, Nvidia tends to stay fairly superficial. Nvidia’s Turing Architecture Explored: Inside the GeForce RTX 2080 had the company presenting DLSS as a feature that enables better performance than anti-aliasing at QHD and 4K, simultaneously achieving better picture quality. It's the claim of higher-quality visuals at faster frame rates that perplexed us most. We naturally spent time comparing the performance and output of DLSS versus TAA (Temporal Anti-Aliasing, a technique for smoothing out the crawling and flickering seen in motion while playing a game) in our first GeForce RTX reviews.
Most recently, Nvidia's GeForce RTX 2070 reviewer's guide explained DLSS by saying, "DLSS leverages a deep neural network to extract multidimensional features of the rendered scene and intelligently combine details from multiple frames to construct a high-quality final image. This allows Turing GPUs to use half the samples for rendering and use AI to fill in information to create the final image." This explanation left us imagining that the graphics processor was only shading part of each frame, leaving the architecture's Tensor cores to reproduce the missing pixels through AI.
Might DLSS more simply be explained as an upscaling technique, perfected through the application of AI? It seems plausible, especially since image processing is one of the most compelling applications of AI. It's also possible that DLSS involves a mix of upscaling, anti-aliasing, and filling in mixing pixels.
The First Visual Cues
Sometimes DLSS looks better than TAA, and sometimes it looks worse. In either case, the technology's output looks very good. Our analysis focuses on individual frames with hand-picked regions zoomed in. But real-time gameplay makes it tough to differentiate between DLSS or TAA at 3840 x 2160, and in certain scenes, artifacts that plague TAA leave the DLSS-based picture unscathed.
Interestingly, we've found that DLSS runs more efficiently at 4K than QHD, yielding a cleaner-looking output. However, when we look at certain image captures, specific hints suggest the picture is rendered at a lower resolution than what is claimed. We were also able to modify the configuration files of Nvidia's Infiltrator and Final Fantasy XV DLSS-enabled demos to run them without AA. This helped immensely with our analysis.
All of our screenshots facilitate comparisons of strictly identical images (aside from some unavoidable variations due to lighting effects). The enlargements are done without filtering to preserve each picture's authenticity. Click to view the images in PNG format (without loss and in their original size).
When DLSS Works Wonderfully
In this picture, it's difficult to distinguish between technologies. DLSS does a great job, and you can even see in the background vegetation that it offers superior image quality compared to TAA. Remarkable.
At this early stage, the Final Fantasy XV demo represents the best implementation of DLSS that we've seen. The Infiltrator demo is somewhat less flattering, though its DLSS rendering also looks great to the naked eye watching in real-time. Again, in some scenes, DLSS is very effective indeed. The following image, DLSS comes close to perfection.
When DLSS Shows its Limits
After zooming in on dozens of screenshots to get more familiar with DLSS and its strengths/weaknesses, we were able to uncover flaws that made us wonder whether DLSS was natively rendering at 4K (3840x2160) or QHD (2560x1440).
We also noticed that DLSS betrays its true resolution on the first frame of every new scene (see above). In the image below, screen captures taken 40 frames later show DLSS smoothing the jaggies with great efficiency. Especially at 4K, the output quality of DLSS is difficult to distinguish from true 4K with TAA applied.
Aliasing is sometimes visible in the middle of a sequence with DLSS active, though, and it persists through the scene. Check out the image below, where jagged edges are more prominent in the 4K DLSS capture compared to TAA at 3840 x 2160.
We wanted to know more about what was going on under the hood to yield such different results. And in the following pages, we finally figured it out...
The Strengths and Weaknesses of DLSS in One Screenshot
In the shot with DLSS enabled, the background and its vegetation look better than the screen captures with no AA or with TAA enabled. But aliasing is more pronounced on the edges of the car. As a final clue, the text on the license plate in our DLSS example reveals a lack of definition compared to 4K with and without AA. In short, DLSS can't always perform miracles.
Note: This story originally appeared on Tom's Hardware FR.
MORE: Best Graphics Cards
MORE: ;Desktop GPU Performance Hierarchy Table
MORE: All Graphics Content
