Aquamark3: Accurate Benchmarking for Old and New (DirectX9) Apps?

"Bug" Or "Cheat"? - NVIDIA Drivers Don't Render The Correct Image

Before we show the performance results of actual cards in the benchmark, we need to take a look at the image quality of the cards and different driver versions first.

The image comparisons we made between the ATI Radeon 9800 and NVIDIA FX5900 cards quickly made it clear that something is wrong with the NVIDIA driver. The lighting in terrain and objects of v45.23, for example, do not work.

NVIDIA explains it's a bug but how this bug got into the NVIDIA drivers is unclear. The fact is that the old driver version v44.03 renders a fully correct image but its performance is very low. ATI and NVIDIA both had worked with beta versions of AquaMark 3 for months, and we even had an early beta version of AM in April. So it's hard to understand how this could happen. Even the new Detonator v51.75 beta driver that was sent to the press last week by NVIDIA, and which was claimed to be the driver to use for DX9 benchmarking like Half-Life 2 and AquaMark 3, renders an incorrect image.

We also tested an older ATI driver (Catalyst 3.4), which was released at the same time then NVIDIA's v44.03. We found the following problems the drivers posed:

• NVIDIA v44.03: Renders a correct image. Faultless anisotropic filtering;
• v45.23: Deterioration of quality. Light on terrain is not displayed. Light on objects is not displayed (sometimes). Shadows are not displayed;
• NVIDIA v51.75: Deterioration of quality. Light on objects is not displayed. Very low anisotropic filtering quality.

The performance numbers change dramatically. From v44.03 to v45.23, the AM3 Triscores climbs 71% and 82% to 51.75. The scores with 8x anisotropic filtering are 70% higher with v45.23 and 92% higher with 51.75 than with v44.03. The anisotropic filtering quality deteriorates very noticeably from 45.23 to 51.75.

We did not find any differences in image quality between ATI's Catalyst drivers v3.4 and v3.7. The performance of a Radeon 9800 Pro 128 MB is about 1.5% faster in v3.7 and this difference can be found in all results and is in the range of natural driver improvements. We only found one case in which the ATI drivers show a lower image quality than NVIDIA's v44.03 driver.