Driver Quirks And Benchmarks
Lately, "application detection" has become the word that makes hardware reviewer's blood boil. Naturally this doesn't have to be a bad thing since it's a good way to solve driver issues in certain games. Since all 3D-engines differ in details, it can easily happen that a driver which works fine for the majority of games causes trouble in one title. The fix normally comes in shape of a new patch for the game or by a fix implemented in a new driver. If done by a driver fix, it will then detect the application and use the new fixed or fitted code.
Even optimizations aimed to achieve better performance are not necessarily a bad thing - if they do not lower the quality of the rendered image. And this is where the trouble begins. Today it becomes impossible to assess the anisotropic filtering quality of a card or driver. ATI and NVIDIA are using a so-called adaptive filtering technology, which means that not all angles of triangles in a scene are rendered at the full quality level or not fully trilinear. This can also differ from application to application and here we are: application detection.
No real trilinear anisotropic filtering in UT2003
Test tools don't help much anymore because they can be detected by the driver resulting in an optimal quality in those test apps. The only thing you can do today is to compare the quality in screenshots but those results are only meaningful for this single application. This way is also very time-consuming and you also have to capture exactly the same frame, which is only possible in a minority of games. It's getting even worse when it comes to FSAA because the image you see on your captured screenshot is not inevitably what you see on your monitor. NVIDIA is using a kind of post frame buffer filtering technology, which was already used by 3dfx's Voodoo5. It takes the samples from the frame buffer and finally combines them in the RAMDAC of the card. The problem is that screen captures can only be taken out of the frame buffer. The technology used by NVIDIA is valid but it complicates reliable screen capturing.
The next problem is the benchmarks itself. Only a little number of games comes with a benchmark function so hardware reviewers always concentrated on the small numbers of available "benchmarkable" titles. So did the chipmakers. They optimized their drivers for those games to the max, which sometimes resulted in benchmark scores that had only little to do with the real performance of the card while playing the game.
The best solution to get over these problems and offer the reader a more meaningful estimation of the performance of a card is to extend the number of games used for benchmarks, to use newer games and to use time demos that are unknown to the driver coders. If a game does not offer a benchmark mode there's the way to use tools like FRAPS or moreBENCH LX to measure the frame rate. We are not aware of any cheating attempts regarding these tools... and every manufacturer trying this will get a very hard time....
