How we computed the GPU index
Last month, TG Daily premiered our dual-core CPU price/performance curves, comparing Intel and AMD models. There, we were able to show that after precipitous price declines last month, AMD recaptured the price/performance lead for "value" processors, although Intel now holds on to the high-performance category, and has the best performing dual-core CPU per dollar overall.
Our GPU index is based on the same logic, though our approach has to differ somewhat: We correlated multiple rounds of benchmark tests over the years, featured in Tom's Hardware Guide. In order to obtain a reliable index for relative performance, we needed to choose multiple tests with objectives that were distinguished from one another, but whose history extended back far enough in time for us to obtain a "1.0" card to base our comparisons on.
This turned out not to be an easy choice. With the features and capabilities of graphics cards having changed radically over just the last few years, it's more difficult than ever before to consider a recent model card in the same category as one made three years ago. As long-time Tom's readers already know, this means the test suites themselves have changed, so a direct answer with regard to how much a product line has evolved quantitatively is not forthcoming. It takes a bit of math, actually...which gives us a good reason to be doing something with these computers other than just playing games.
Our index system, therefore, is the 128 MB Nvidia GeForce 5900, which launched in 2003. For the purposes of these comparisons, the 5900 always scores a 1.0. All performance indices are therefore relative to this card. Does this materially affect our judgment with regard to either brand, Nvidia or ATI? No, because all three test suites we chose for our overall performance index judge both brands' performance relative to one another as well.
The three test suites for our performance index cover categories which pertain to three-year-old models as well as today's models, which does indeed limit what capabilities we cover. For example, high dynamic range (HDR), or the ability to extend the range between the brightest brights and darkest darks - which is a feature not only of modern cards but of newer games such as Serious Sam 2 - doesn't apply to the 5900, nor does Shader Model 3.0. Of course, we could have chosen a newer card as our baseline index, but like the case of a Doppler radar whose granularity becomes weaker the closer it is to the center of a moving storm, index values for other cards in the vicinity of the baseline index card would have been less informative. We needed a card that was old enough that we were no longer too interested in other cards of its same era, but new enough that we didn't place a gulf of capability between our baseline card and our test subjects.
The benchmarks we chose were as follows:
- F.E.A.R. at 1024 x 768 resolution, with 4x anti-aliasing and 8x anisotropic filtering, and which tests the card's abilities with Microsoft DirectX drivers;
- Quake 4 at 1280 x 1024 resolution, with 4x anti-aliasing and 8x anisotropic filtering, and which tests the card's rendering capability with OpenGL;
- 3DMark06, which is the leading synthetic benchmark.
For this latter test, the problem is as obvious as the big, giant year in its name. We don't have test results for many cards which premiered two years ago, for the 3DMark06 test; likewise, we don't have 3DMark05 results for some of the newest cards. But we do have a certain swatch of tests for which we have both sets of results. So we had to do a bit of mathematical modeling: Specifically, given the older cards' 3DMark05 scores, we estimated what their 3DMark06 score would be. (We attempted the converse approach to see whether it was more reliable, estimating the 3DMark05 score for one card for which we already had the score. We then estimated the 3DMark06 score for a card whose results we also knew, and concluded the latter estimate was more reliable.)
The relative capabilities of each card for which Tom's has obtained results were scored relative to our figures for the Nvidia GeForce 5900. (By the way, our estimated 3DMark06 score for the 5900 is 126.) So a 512 MB ATI Radeon X800 XL with a Quake 4 score of 6.92 in our tests performed 692% better than the 5900 at rendering OpenGL scenes at 12x10 resolution. Its overall performance index is 7.27, which is the average of the three test scores put together, each test counting 33.3% of the total.
Here's something else that readers yesterday noticed: We don't have scores yet for all the cards produced, including both old and new cards in both product lines. As Tom's Hardware Guide continues to compile more comprehensive data, we'll incorporate new numbers into our charts. Wouldn't these missing cards affect the projected price/performance curve one way or the other? If so, we feel only slightly, mainly because the existing data gives us more than enough information to actually project these curves. It's like a very high percentage polling sample for a precinct of fewer than a hundred voters; if one or the other candidate is in the lead by 20% or more, you can pretty much project the winner with 90% of the precincts tallied.
