Sign in with
Sign up | Sign in
Your question

GPU Normalized Perrformance Rating charts

Last response: in Graphics & Displays
Share
May 3, 2008 10:41:33 PM

I created some supplemental charts for Tom's Hardware GPU and CPU charts.

http://emsai.net/reviews/

Who knows, maybe Tom's will add something similar too. At the very least they got easy access to a lot of cross-referencing data over the years. ;) 

(from my journal entry today)
A new Reviews section has been added, and two very new cool additions under that are the CPU and GPU normalized performance rating charts, with a very impressive upgrade adviser integrated into the chart.

The Normalized Performance Rating (NPR) charts uses the data of Tom's Hardware CPU and VGA charts, 31 of the former result tables and 43 of the latter result tables was painstakingly normalized, merged, verified, and can now be seen in all their glory. CPU Chart

These charts was made out of my own need as Tom's Hardware lacked anything similar, sure there is the FPS table in their VGA chart, and the price/performance index in their CPU chart, but neither was suitable for my purpose. I needed to compare all CPU's and GPU's against a baseline so that I could check possible upgrade choices, and see a range of potential upgrades.

It is also very interesting to see how similar the performance increase are for both CPU's and GPU's, and by looking at the charts it is obvious that new does not always equal better. Since my charts are based on re-purposed data from Tom's Hardware, my two chart tables are also available as CSV files if that is of any interest to anyone.

It would be really nice if the industry used a normalized performance rating system like this as it's so easy to see what hardware has x amount of improved performance over the other. Leaving the consumer to worry about cost and features only. Now if game/software developers hopped on board that would be even better.

Yes I know that there is the Windows Performance Index, but it's not as fine grained as my NPR (only two fraction points are shown, the CSV itself has up to 15), you may also think that there is no point with all the benchmark and chart sites out there. But look at my charts and you see it's all floating point math which can so easily be extrapolated, secondary baselines like the upgrade adviser does is very simple and fast to implement.

I will update these charts now and again, and if popularity gets really high I will obviously update them more frequently. I may or may not add older CPU's and GPU's, it all depends on if I can find some solid benchmark data someplace that I can normalize then re-normalize against the NPR baseline and finally add them to the charts.

Enjoy the new Reviews section, I have a few future plans that might be just as interesting.
a b U Graphics card
a b à CPUs
May 3, 2008 10:53:27 PM

Good work. You might want to change highlighted colors to some thing different (esp. the brown)

Question: How can HD2900 XT CF be better for (performance vs. value compared to a 8800GT in SLI?
May 3, 2008 11:19:23 PM

I thougth it was a good Idea until the chart suggested swapping 8800GTX SLI for HD2900 CF, then I realized something was seriously messed up.

Delete the HD2900 CF from the top and the chart will be good to go.

Also change the brown.
Related resources
May 3, 2008 11:24:49 PM

Thats some nice charts you got there. I again would mention the 2900XT's. They would be at the top if you took into account overclocking aswell. They do scale quite well like that.

I don't mind the brown it makes you look straight at the green cards instead, which is what you suggest anyway :) . All in all i nice change :D .

*EDIT* Awww I can't find my 3870's in CF on there...
May 3, 2008 11:51:02 PM

Well, the NPR is twice as high, so unless Tom's benchmarks was wrong it performs twice as good. As to value, that depends totally on the prices.
Tom's do not have a Performance/Price index for GPU's. (they do for CPU's though)

But it looks correct to me. You may have gotten it the wrong way though.
HD2900 XT CF = 7.79
8800 GT SLI = 4.80

Yeah it is a almost doubling in performance, but I assume (didn't check any stores for price) that it is very expensive compared to the 8800 GT SLI?
If that is the case then the logic used works as intended.

If you click on 8800 GT SLI you will see that the OC is brown and not really worth upgrading to, and the HD2900 XT CF is blue indicating that such a dramatic upgrade may be too costly. (from 8800 GT SLI to HD 2900 XT CF)

I may tweak the advisor calculations if it turns out it's very wrong but it seems to be pretty good so far.

As to the colors, that was a huge pain too. I tried darker color variants of those and that looked even worse. If I find some better matches I'll use those obviously. But those are among the better CSS standard named colors, was hoping to avoid HEX colors as those can tend to look very different on certain displays.

PS! I did ponder adding a price/performance type info as well, but prices change, varies from country to country, and unless the price inded is automated it would be a pain to maintain. Tom's has a automatic one as one of the tables in the CPU chart. None for the VGA chart yet though.

Besides I don't want to replicate what Tom's are doing, this is just a supplement for quick lookup and comparison against the default baseline or the baseline you get when clicking on the charts. Which is also why I link back to Tom's so people can see the various tests, and check Tom's for prices etc.

As an example. Click on 7600 GT, as you see the 8800 GTS is in the middle of the green, it should hopefully be a good performance upgrade while not causing you to loose your shirt due to cost. Lower mid end to higher mid end/lower high end upgrade in other words.

Upgrading from 8800GTS is not so easy though, or rather more limiting. Clicking that and looking at the chart, I'd say that getting a second 8800GTS and setting up a SLI seems to be the best choice in that particular case.

It will be really interesting when the next few cards are launched, that should add more meat to the top of the chart a little.
May 4, 2008 12:05:22 AM

"White is your current selection. Red means you will get less performance with those GPU's. Brown means improved performance but not enough gain compared to the cost. Green means a 2-3 NPR increase and is the advised upgrade range and hopefully affordable. Blue means too much performance increase thus potentially too expensive."

And the 2900XT is King... FTW?
May 4, 2008 12:06:17 AM

I just double checked Tom's tables. It seems the HD2900 XT CF does extremely well in the synthetic benchmarks, and pretty ok in the game benchmarks.
This could be why Tom's has that FPS of all games table rather than a total performance etc index.

So either ATI is cheating on synthetic benchmarks or that card is just damn good on those benchmarks. My NPR charts are still correct though. The normalized summing of all of Tom's tables. I trust the tests Tom's do, synthetic benchmarks may be more dramatic than custom ones like the game FPS tests.

PS! If you want a non synthetic NPR just take the FPS table index in To'ms chart. decide on a baseline, then do a lil math to normalize all FPS to the baseline. Then again, it might be just as easy to just look at the FPS and use that as it is when making a decision if you are a avid gamer.
a b U Graphics card
May 4, 2008 12:09:54 AM

Wish I had seen this thread before posting in the cpu one; won't bother retyping it all but most of my comments were GPU related - link: http://www.tomshardware.com/forum/250070-28-normalized-...


Yeah, crossfire 2900XT > 8800GTX SLI is way off. I commented on this in your CPU thread. Where it is going to count is high resolution with eye candy (fsaa/af).

Even Tom's fps totals won't work for usefulness. If a bunch of 1024x768 no eye candy victories, or 3dmark victories, get averaged in with what's really important, it will severely and negatively impact the results and usefulness. A bunch of 120 fps vs 110 fps victories could negate some 20 vs 40 fps victories where it counts.

Like I said in that cpu thread, the charts can be very misleading if misused.

May 4, 2008 2:08:47 AM

Well I certainly feel stupid now. It turns out that the GPU datafiles here had a duplicate of HD2900 XT CF, originating from Tom's own charts where this card is dual listed for some odd reason. The CPU charts seem unaffected, at least nothing as odd as this.

The csv builder has had a few checks added and a new GPU chart generated, after checking all benchmarks again, ratings now match what is expected.
May 4, 2008 2:13:03 AM

Rescator said:
Well I certainly feel stupid now. It turns out that the GPU datafiles here had a duplicate of HD2900 XT CF, originating from Tom's own charts where this card is dual listed for some odd reason. The CPU charts seem unaffected, at least nothing as odd as this.

The csv builder has had a few checks added and a new GPU chart generated, after checking all benchmarks again, ratings now match what is expected.

It's because Tomshardware's new chart itself is lousy, and not your fault and whatnot... :p 

How about something like this?
http://www.guru3d.com/category/vga_2/
May 4, 2008 2:29:11 AM

Hehe, thanks but I can't directly use that. What I do with Tom's is choose a benchmark, display all entries, mark all the benchmark related text and do a simple copy'n'paste into a text file.
I then locate my baseline in each of the 43 text files I made, note it at the top, then add + or - indicating if a high score is better or if a low score is better. then I feed them all through my csv builder tool I made.

Reason I feel stupid is I should have seen it when manually copy'n'pasting the text as that card was listed twice right next to each other. *slaps head*

Ideally there should be more sites like Tom's (that have nice big charts), but there aren't, or there are but the tests are small and thus not imperical, or they are tainted by user submitted data. (there is one huge chart site out there but it's data is not as reliable as Tom's due to that) There are some ok ones but the tests are small and the results are in images so not that nice on my sanity to merge.

The best would be to merge like I did, not just Tom's charts but charts from many other sites, to get a sort of superchart, the varying types of testing setups and tests would actually make the NPR chart even more correct. (as it would reflect the very varied mix of consumers setups and uses better)

But don't worry, I'm sure Tom's will keep improving their charts, and hey maybe my 'lil NPR charts will be redundant in the future as Tom's might have something similar. *pokes Tom's DB guys*

a b U Graphics card
May 4, 2008 2:10:23 PM

For a quick lookup chart like you say, it's a nice concept. And I agree people need to use some sense in what they do with the results. I do think there should be a red flag of caution thrown up to weighing synthetics/games/aps together and coming up with an overall ranking.

My problem stems from the Tom's charts themselves and how people use them. Nice tool for quick reference, but easy to misuse. All too often (bad) recommendations are made in the forums and when questioned people point to the charts. But the recommendations were being made highly based on low resolution scores in old games without fsaa. You can't keep a running log of old performance testing and expect to keep things up to date. Best available options change, Games change, so do drivers. And there very often is no clear best solution for everyones needs, so overall total scores really mean little if the people don't do the research to check them vs their own specific current needs. The GF7's for example can be very good for those who play old games(like the charts show), so why upgrade. But I pitty the fool who looks at the charts and then buys a second costly 7950GT now thinking his SLI combo will give him > HD3870/9600GT level performance in newer titles. Then they fire up UT3, Crysis, or Oblivion and come asking why there 1600x1200 performance is so bad. The charts have the 7950GT SLI > HD3870 and crossfire X1900XT even.

Right now, some of the best GPU options for people are not on the charts, and so many of the options that are the charts look way better than they currently are if you look at specific new games played at a specific gaming resolution.
May 4, 2008 3:28:56 PM

It is possible the NPR chart will deviate from Tom's from now on as individual cards are added. New additions will be NPR based.

This means a baseline in card comparisons will be found, if that baseline exist in the NPR chart someplace adding it is as easy as multiplying the difference between an existing card and a new one, and there's your NPR for it. So there is no need to recalculate the entire thing again.

This also means the NPR chart is test independent.
The "trick" is simple, if card B is x% more powerful than card A then it will always be x% more powerful than that card and it's place in the NPR chart is clear as I have no plans to change the NPR chart's baseline (which is the 7600 GT at 1.0 NPR) maybe in 10 years a new baseline is needed as the highest NPR's in the chart is probably in the hundreds or thousands by then.

Check by the chart in the future weeks/months, or just subscribe to the site's RSS feed as news of updates will be added to the RSS feed, the site is rather calm so at most there will be one journal entry per month.
a b U Graphics card
May 4, 2008 5:13:14 PM

But, most review sites will no longer test using a 7600GT, so it will be hard to get future results. And if the system specs, tester, and test itself differ in a new review, you obviously can not compare the results (directly) to the old 7600GT numbers. Tom's charts itself actually does this to a small degree as test systems do vary when the reviews are done over a period of time. Not to mention the test suite of games is far from what most gamers/reviewers now would want to use to base performance. (games from 2004-2006). Oblivion is a keeper, the others??? Worse too, would be assuming Card A = 1.14 of the 7600GT in these old games, so we can estimate the 7600GT performance in a game it was not tested in the future. That's already way off in current titles, and will continue to be so in new titles. Far worse(but a good example) would have been looking at the GF-FX series in DX8 and guessing it's DX9 performance down the road. ;) 

I can't see how comparing to a 7600GT in overall performance will really benefit anyone (apart from other GF7 series owners) honestly, whether it's Tom's charts or something else. It's a funny card who's architecture was very good back then, especially without fsaa. But it is not very good (compared to it's former equals) in current shader heavy games. Does this make sense? Let me explain by example:

Take the 7600GT currently, in the tests that were used to base those number on.

Now, 7600GT = 1.00, HD2600 pro = 0.91, HD2600XT = 1.14, 7950GT = 1.72, X1950 pro = 1.53, 8800GTX = 3.33 (taken from your chart)

Now look at just one big name current game not tested in the performance charts (happens to include the 7600GT) : http://www.anandtech.com/video/showdoc.aspx?i=3128&p=4

@ 16x12 res (ease of reading chart)
7600GT - 20.4 fps
2600 pro - 28.3 fps (+ 38.7%)
2600XT - 42.6 fps (+ 108.8%)
X1950 pro - 43.8 fps (+ 114.7%)
7950GT - 35.0 + (+ 71.6%)
8800GTX - 100.4 (+ 392.2%)

As you can see, 7600GT based comparison using Tom's charts is way off base for a title like UT3, apart from the GF7 architecture (7950GT). And these results would vary too if we look at a different resolution or enable fsaa. Possibly differ again quite a bit if we use current latest retail patched game and newest drivers. Same will happen with Oblivion (in outdoor foliage GPU demanding areas), Crysis, NFS:Carbon, COD4, etc.

I hope you see this as constructive criticism. You are pouring a lot of time/energy into this, I'd like to at least make you aware of these points(if you were not), sooner rather than later. The way I see you are in for a huge project that will have a hard time accurately portraying performance in the various different current /future games.



May 6, 2008 7:25:35 AM

The NPR charts for CPUs and GPUs have been updated.
7 new CPUs (AMD), and 6 new GPUs (ATi and Nvidia).

The charts can be found in the Reviews section as always.
http://EmSai.net/reviews/

New additions for GPU's are:
ATi HD3870 X2
Nvidia 9800 GTX
Nvidia 8800 GTS 512MB
Nvidia 9800 GX2
Nvidia 8800 GT 256MB
Nvidia 8800 GT 512MB

The CPU and GPU charts has been cleaned up as well,
OC variants has been removed, so has various stepping variants,
the latest stepping variant was kept in case of duplicate entries.

The information in the charts has changed to, no more Megahertz,
instead it's all about Cores now as this does not just affect performance but multitasking ability as well.

A feature column has been added as well as the features a GPU or CPU is just as important as it's performance.

The header of the Feature column has two lines in it called All: and None: this is a simple way to inform shared features or non-existent features that applies to all entries in the chart.

As for the GPUs no cards support OpenGL 3.0 yet, in fact none support OpenGL 2.1 yet but the upcoming cards should support GL2.1 it seems.

DirectX 10.1 and Shader Model 4.1 support is sparse so far but upcoming cards should all support it hopefully.

Enjoy the charts and as new CPUs and GPUs are released I'll update the charts as often as time allows.
May 6, 2008 7:40:57 AM

Thanks for your feedback pauldh.

But the 7600GT was only used as a baseline for the initial chart building.

Individual CPUs and GPUs that are added to the NPR charts do not use the 7600GT as a baseline, instead individual additions use whatever other GPU is used in the test data that happen to exist in the NPR chart. THat becomes the baseline for that.

Like the current update, the 8800 Ultra was the baseline when normalizing the new additions, as that existed in some of the tests and in the NPR charts as well.

Once the tests was summed and normalized against the 8800 Ultra as a baseline, it was simply a matter of taking the 8800 Ultra's NPR value from the chart and multiply it with the normalized result of the new cards, that result was then simply inserted into the chart.

The fact that the 7600 GT is 1.0 and thus the baseline does not matter. I could easily change this to something else or a different card.

As long as at least one of the cards in the NPR chart also appears in the comparison tests of a new card's benchmarks and the test is conclusive enough, I can use it and add it.

It's all simple math.

I am not limited to just Tom's benchmark tests either, I could just as easily get the data from elsewhere.

After all, within a single test, if one card has twice the performance of another, then it has twice the performance of the other. And although driver improvements can slightly skew results it is usually not that dramatic *knock on wood*

The beauty of the NPR chart is that not only can you see the NPR rating against the baseline but thanks to it being floating point math (see the CSV for the full length numbers used) the NPR rating is actually relative between all cards.

The only time I'll struggle is if test benchmarks does not include a single card from the NPR. In that case I'll have to start hunting to find benchmarks that include that card, and once that it added I can go back to the the previous benchmarks as I now have a reference point/temporary baseline.

Adding new entries to a NPR chart is done using a notepad and a calculator, by hand. It's as simple as that. ;) 
June 16, 2008 7:45:37 PM

Added the new Nvidia GeForce 280GTX and 260GTX cards to the NPR chart.
http://emsai.net/reviews/gpu/

Double checked with AnandTech's tests and despite those having less tests and no synthetic benchmarks, Tom's tests and Anandtech's are roughly within 2% of each other. Which is as good as it gets as far as margins of error goes I guess. Making Tom's tests and thus the NPR chart entries as well as close to imperical as you can get. I'll obviously correct the NPR chart if more large tests or re-tests shows any large discrepancies obviously.

Note! For those that do not like synthetic tests, I find them vital as they show the relative performance of hardware against other hardware under ideal or dedicated tasked use. For most users this has no meaning, but as GPU's are sort of becoming a secondary CPU now, those synthetic tests are actually valuable. Another thing is that real world tests tend to vary, while lab tests are more flat, sum the two and you get rather solid numbers.
June 22, 2008 8:20:29 PM

The GPU chart has been updated, Nvidia's GeForce 9800 GTX+ and AMD (ATi)'s brand new Radeon 4850 graphics cards have been added along with the older Radeon 3650 and 3450 that was missing from the chart. As usual you can find the charts in the Reviews section. http://emsai.net/reviews/
!