PREEDIT: Gah, dear toms hardware, please make it so that when I don't choose a sub-category for my post and then select 'submit' I don't have to re-enter my entire comment text!
Ok so here (again) is my suggestion for benchmark results, which I've been whoring round a few sites, designed to give a fuller picture of component performance than mere minimum/maximum and average results that are fairly common now.
It's the 'sustained performance' chart.
What it shows is the % of time a certain level of performance was achieved during a single benchmark (not an aggregate over several different benchmarks). It might not lean itself towards some benchmark types but should be easily applied to things such as gaming/FPS results. I don't know how easily these stats could be gathered but I'd have thought there must be some FPS tool out there that can output a log of FPS figures which could be hauled in to a spreadsheet for analysis.
Consider 2 graphics cards (I made up) the XYG 260GT and the ABC 3900S.
I dummied up some benchmark results on a spreadsheet. I don't pretend that they're realistic (I just used random number generators and formulae and crazy math functions) but they do make a point and show how these charts might be useful.
The figures show that both complete a benchmark with the same maximum and minimum FPS of 10/82 FPS, the XYG has an average FPS of 47.38 and the ABC 44.91.... so the XYG is the clear winner, right?
Let's take a look at the sustained performance chart:
[<edit> further explanation of how the graph works
If you look along the bottom it shows a certain FPS. The left axis shows the % of time that FPS or higher was achieved.
So if the line for a card/chip passed through the point that corresponds to 30 FPS at the bottom and 90% on the left, it would show that when the benchmark ran, the card achieved 30FPS for 90% of benchmark.
In other words, it only dipped below 30FPS 10% of the time.
So if the benchmark ran for a total of 100 seconds, for 10 of those seconds it ran at less than 30FPS and for 90 seconds it was producing greater than 30FPS.
Make Sense?
</edit>]
Here we can see that, despite it's higher average FPS, the XYG barely manages to limp above 30FPS 30% of the time, spending 20% of the time below about 19 FPS. Perhaps the problems is with a certain graphical effect or the XYG can't cope when a scene comes particularly crowded as the manufacturer, for some unknown reason, despite giving it 1Gb of superfast DDR5 memory, decided to lump it with a 128-bit data bus. Of course such a design mistake would never happen in reality
![:sarcastic: :sarcastic:]()
, but you get my point.
Conversely the ABC has a much smoother performance curve, maintaining 30+ FPS 90% of the time.
So ... which would you rather game with? Ok, I can hear cries of 'neither' from the hardcore enthusiast '100FPS or die!' gamers, but let's pretend this is a Crysis 4 (TBA) benchmark, so 30+ FPS isn't that bad.
Meh, just a thought. Any takers/comments?
It's not uncommon to read a review where the reviewer rates a card highly, making comments about the 'overall experience' or some such, despite seemingly unimpressive benchmark scores, or denigrates a product for 'choppy gameplay'. These charts provide the data to back such claims up.