3D Winbench 98 - Only a Misleading Benchmark or the Best Target for Cheating ?

NVidia Interview

These are the answers of nVidia , the company that currently sells the most 3D accelerator chips.

1) I know that you guys at nVidia are working like crazy, I once spoke to Dwight Diercks at 3 a.m. PST. Did you try getting your new beta OpenGL ICD out before the i740 launch on purpose or didn't you care about Feb 12 ?

We work hard because we set the difficult goal of delivering the best products in the world. When we specified our OpenGL implementation we wanted it to be fantastic. We could have cheesed a miniport driver but we wanted to deliver a full ICD. A full ICD is a big job. It's taken us a little longer than we had hoped but we put out the betas as soon as we had something we were proud of.

2) What are nVidia's thoughts about the i740? Do you think it's gonna be a tough competitor?

The i740 is slower than the Riva 128 and almost a year later. It will probably be a tough competitor for the companies that don't have a great 3D product and who are trying to sell on price.

3) Who does nVidia see as most important competitors anyway?

We try to be our own most important competitor. Our goal is to obsolete our products before someone else does. In the process of doing that we will deliver truly state of the art products for our customers. We've set the goal of delivering an entirely new generation of graphics capability and performance every 12 months. We have no fewer than 3 separate development teams operating concurrently to obsolete each other. Imagine 3 internal competitors that know exactly what each other are doing! That's the toughest competition anywhere!

4) How important is 3D Winbench for nVidia and how important do you think it is for the 3D card industry?

Every developing marketplace needs some way of measuring the goodness of a product. 3D Winbench is a good start at a metric for measuring goodness of 3D cards and systems. Is it perfect? No. Is it important that the industry support such a metric? Yes. Otherwise, it's very hard to get a quick understanding of a product. But it's just that - a *quick* understanding, not a *complete* understanding. You wouldn't buy an automobile based on the horsepower of the engine. Yet knowing the horsepower of the engine does give you a quick understanding of the auto. And that's the value of 3D Winbench.

5) Is it correct that drivers can be easily optimized for one application, e.g. 3D Winbench 98?

Yes, drivers can be tuned specifically for a benchmark. But trying to fool customers is a terrible practice and just won't work for long for any company that tries it. Real customers run real applications. 3D Winbench is just an approximation of a whole bunch of applications. It's a good first step but by no means as good as testing a bunch of real applications.

6) Has nVidia optimized the drivers of the Riva 128 for 3D Winbench98?

Yes! But that's the wrong question. What you should have asked is whether we have optimized our drivers to exaggerate our performance on 3D Winbench98 relative to Riva 128 performance on real 3D applications. The answer to that is an absolute *NO*. We have certainly used 3D Winbench as one of the tools for improving our drivers. 3D Winbench does things with D3D that no existing applications mimic. Any company that has not extensively studied 3D Winbench is arrogant and has likely produced a set of D3D drivers that are far less robust than they should be. Similarly, any company that has high 3D Winbench scores with disproportionately less performance on real applications should be viewed with _great_ suspicion. NVIDIA is extremely self-conscious about Riva 128 performance and quality on real applications and has made a huge investment in testing and quality assurance. Looking great on benchmarks while looking slow on real apps is a slimy way to build a business. We won't do it.

7) If you were a normal home or office user, would you base your buying decision on the 3D Winbench scores of a 3D accelerator?

No. 3D Winbench would be just one data point that would help me make a decision.

8) If 'no', what would you base your buying decisions on?

For 3D applications, I'd also like to understand the performance on a mix of real world applications that seem similar to what I will use. I'd also want to know how state of the art my card was because that would give me some idea of how long it would be before I would have to replace it. So I'd try to understand the basic technology beyond 3D processor speed: AGP, amount of memory, graphics memory bus width, and support for high resolutions. I'd also want to know it's 2D performance and whether the technology was coming from a reputable company that would service their products. I'd also want to know the price and what software came bundled with the card.

9) Could you imagine that graphic card manufacturers would try and influence the buying decisions of users by optimizing their drivers particularly for 3DWinbench 98?

Of course. Look at what happened with Dhrystone. That's why real applications performance must be measured in addition to 3DWinbench to get a complete picture.

10) What are the strengths of the RIVA 128 in your opinion?

Great 2D and 3D performance in a single card. Extraordinary performance upside due to the 128-bit bus. Riva 128 based cards will run new high resolution titles well at 800x600 and beyond. Plus, the true power of the Riva 128 is just beginning to be tapped. Our next software release will *dramatically* improve performance on both D3D and OGL. Riva 128 based cards scale in performance well up to 400 MHz on P2 systems. Throw in a world class OpenGL implementation (full ICD!) and then be amazed at how little it all costs.

11) What are the weaknesses of the RIVA 128 in your opinion?

For a few people, lack of 8M memory support for 1600x1200 true color. For most, es macht nichts. (Wow, Kevin shows his German roots!!! 'es macht nichts' means 'it doesn't matter')

12) What's the next chip of nVidia gonna be? When will it be launched?

Even better than the first...but that's all I can say for now :-)

13) When will the final OpenGL ICD be finished?

The OpenGL ICD is in testing now and will ship within 60 days.

14) Do you guys ever sleep?

Generally very little. Our CEO never sleeps and we all aspire to match his abilities.

Thanks a bunch Kevin for these detailed and very honest answers!

Kevin Schuh, nVidia