Benchmark Results: 3DMark11 (DX11)
I recently spent some time on the phone with Nvidia discussing 3DMark11. It seems the company doesn’t like this benchmark because it doesn’t reflect what it considers being the future of game development—that is, increased use of geometry to improve realism.
No doubt, the emphasis in 3DMark11 is on lighting—a particular DirectX 10 shader, specifically. However, I also had a very fair and balanced call with Futuremark, which says it designed the benchmark with the right amount of tessellation optimization. Throwing gratuitous geometry at a scene has diminishing returns on visual fidelity, but it does have the potential to really hammer the performance of entry-level and mainstream boards. And while the first graphics test in 3DMark11 doesn’t touch geometry at all, the second, third, and fourth do incorporate what we’d consider a reasonable amount of tessellation—that is to say, there aren’t any scenes with triangles so large that we think to ourselves, “wow, that was a missed opportunity to add geometric detail.”
Of course, Nvidia argues that a synthetic like this should be reflective of the future, not just today’s titles. Futuremark counters that it is—at no point will game developers want to add complexity for complexity’s sake.
At the end of the day, what matters most is how reflective 3DMark11 is of a greater sampling of real-world games. And when you look through our results in today’s DirectX 11 games (everything we test here today, except for World of Warcraft, is DX11-based), you’ll notice that 3DMark11 is actually a little bit off for whatever reason. The most notable disparity is between the GeForce GTX 560 Ti and Radeon HD 6870. The AMD card is scored ahead of the Nvidia board here, but in all other real-world cases (except F1 2010—more on that in a bit), Nvidia lands ahead. The GTX 470 also seems disproportionately lower here than it is elsewhere against the Radeon HD 6870. And even within AMD’s own lineup, it’s interesting to note that the Radeon HD 5870 beats its 6950 in a number of benchmark situations, though 3DMark11 shows the opposite by a substantial margin.
Perhaps 3DMark11 is more representative of the future than Nvidia thinks, if the 6950’s dual tessellation engines are helping keep it that far ahead of the 5870. Or maybe its reliance on DirectX 10-class lighting shaders only serves to emphasize AMD’s strong DX10 performance as Nvidia’s arguably more future-looking architecture is made to suffer. In either case, don’t base your purchasing decisions on the results of a synthetic alone. The folks at Futuremark made an effort to factor out the politicking that goes into heavily IHV-influenced games, but in doing so disconnected the results you’ll see from more…shall we say, affected real-world titles. There’s a place for it in our benchmark suite, but I’d be pretty ticked to see a 3DMark11 score from Tom’s Hardware on the box of any graphics card, used as a marketing point. It simply cannot be digested at face value.
