3DMark06 Under the Magnifying Glass

The "Revised" Engine

According to Futuremark, 3DMark05 used a new rendering engine which acted "more like a game engine than the one we used in 3DMark03." This version got more out of Microsoft's DirectX 9 standard, as 3DMark03 was only an introductory version, using DirectX 9 features in a limited capacity.

3DMark05 cast off shader Model 1.x, and required that all graphics hardware have DirectX 9 with full support for at least Shader Model 2.0 for vertex and pixel shaders. When Microsoft released the 2004 DirectX 9 update to 9.0c, which brought in Pixel Shader 3.0, Futuremark released patch 1.2.0 with libraries that conform to the change with other minor updates and support items. The Futuremark whitepaper on 3DMark05 states that "the engine in 3DMark05 dynamically builds shaders for each material in HLSL (High Level Shader Language) format. These shaders are then runtime compiled to best fit the installed hardware", or coded best to suit the cards they are running on.

Futuremark used Light Space Perspective Shadow Maps (LiSPSM) to create perspective shadow maps. This technique was one of the first approaches to benchmarking with real-time dynamic shadows enabled. However, this approach of a per-hardware design took some criticism from the community, because it was a deviation from previous generations, as was the case with shadow rendering paths. 3DMark05 used depth shadow maps called perspective shadow maps (PSM, a variation of Projection Shadow Map), since there can be issues with shadow projections (self shadowing). The revised version corrects this.

The fact that these shadows are rendered using 2048x2048 depth maps is not where the issue lies; the discrepancy appears in the implementation of more than the rendering path depending on the hardware as described above. Shadows can be rendered in R32F format, but if there is hardware support for depth stencil textures (DST), an alternate D24X8 depth map of the same size is used. Stencil textures are currently supported on both types of hardware, but Nvidia has a special depth format, Percentage Closer Filtering (PCF). ATI does not support PCF, and this meant an acceleration path advantage for Nvidia hardware in the game tests.

The true impact is not that significant, but Futuremark states in its whitepaper that one "could argue that the DST and hardware accelerated PCF implementation vs. the non-DST and point sampling code paths do not produce comparable performance measurements, since the resulting rendering shows slight differences. 3DMark05 was designed with the firm belief that those two are indeed comparable, and in the fact that it is the right way to reflect future 3D game performance." In its defense, Futuremark has made it possible to force all hardware to run the same way, if the user chooses to; the problem is that most users don't. Most scores that are reported are based on the default settings, as that is the easiest way for various parties to test different systems while maintaining identical testing parameters.