Aquamark3: Accurate Benchmarking for Old and New (DirectX9) Apps?

Worth The Wait: Realworld-DirectX9-Benchmark AquaMark 3

AquaMark 3 (www.Aquamark3.com ), by Massive Development, is a new benchmark tool to test the capabilities of graphic cards. Massive (www.massive.de ) has a long experience in shader programming - in 2001, they published the game Aquanox, which was one of the first games using DirectX 8 shader effects. The release was accompanied by the benchmark AquaMark 2. In the end of 2002 they released their new game "Aquanox 2 - Revelation" in Europe. After a couple of beta patches, the game was finally upgraded to DX9 level in April 2003 with the patch v2.159. The international version of the game, released this summer, is based on that level of their so-called "Krass 3D Engine" (www.massive.de/english/technology_eng.html ) and has been a full DX9 game from the beginning.

The just-released benchmark tool AquaMark 3 (AM3) uses the latest version of the Krass engine. The difference between AM3 and other benchmarks, such as 3Dmark, is that it is based on the released game Aquanox 2, meaning it's not a synthetic benchmark. It uses a very complex 3D environment, compared to Aquanox 2, which results in a 3x to 7x higher graphics load.

Massive stresses that AM3 is a "real world DirectX9 benchmark" and not a synthetic test because it is based on a successfully used game engine. This also means that the engine needs to have compromises for compatibility with older graphic cards to accommodate a larger user base, such as those with older DirectX8 compatible cards. So a real-world gaming engine

must include accommodations for older and the newest of new cards.

Not all pixel- and vertex shaders, for example, are based on DX9.

The engine uses a lot of lower shader versions as well, including:

• Vertex Shader 1.1 (DX8): 190 (up to 70 instructions per shader)
• Vertex Shader 2.0 (DX8): 3 (up to 50 instructions per shader)
• Pixel Shader 1.1 (DX8): 50 p to 13 instructions per shader)
• Pixel Shader 1.4 (DX8.1): 5 (up to 13 instructions per shader)
• Pixel Shader 2.0 (DX9): 4 (up 40 70 instructions per shader)

This also means that AquaMark 3 is not a pure DirectX9 benchmark. But Massive explains that offering a pure DirectX9 was never their goal. They stress that they use what they call a "real world 3D game engine."

No game developer can afford to only support the latest graphics card technology. That's the reason why they use a lower shader version for an effect whenever possible. Massive explains that it's no problem to define all shaders as DX9. But this would make the benchmark synthetic.

They also say that not all shaders necessarily need to be based on the highest shader level to run fast. Lower can be better sometimes. One example is the particle effects in their game. It would not make sense to use shaders to create them. They run much faster when implemented by DX7 fixed function instructions. In the end, the average use of PS2.0 in the rendered image is 30%.