We test all cards at all load levels at 72°F room temperature. At least one hour before the test, all cards are moved to the testing room to make sure they are at room temperature when testing begins. After powering on the test rig, we wait for 15 minutes at idle load until temperatures stabilize before we start measuring. We log the temperatures using MSI Afterburner.
In order to measure the temperature under load, we use the Bitcoin mining application (GPGPU) or, if the card is incompatible, a pre-programmed Perlin noise loop from 3DMark Vantage. This typically generates very high FurMark-like loads, but, unlike with FurMark, the drivers don't throttle the performance (and, indirectly, power draw). Thus, the temperatures we see are worst-case, non-throttled temperatures.
Why don’t we just measure the card temperature in a demanding game? The answer is simple: even a demanding game may result in an inconsistent load if we're not diligent about recording the result at the same time. There are also games that generate extremely high frame rates while displaying menus, and the resulting power draw may not just match, but actually exceed, the power draw seen in a Metro 2033 loop and even approach the power draw of a Perlin noise loop. With GPGPU, you know how hot the card will get in a worst-case scenario. Also, the number of real-world GPGPU applications is increasing, so the extreme load numbers will become more and more relevant in the future.
We conduct the temperature measurements at the same time as the power measurements, which are discussed on the next page.
While I love the new techniques, using BITCOIN to bench GPUGPU performance instead of Folding @ Home? Um, okay.
Still with the bar charts? Would *love* to see scatter plots with price/score on the axes... So much more useful in picking out a card.
Man the charts are dying to be updated. Too bad it isn't done more often since it takes a lot of work.
You havent added how many cheese wheels it can run in skyrim as a benchmark... wth?
... i like the pile of card's @ the end of the article.... a beautiful pile...
While I love the new techniques, using BITCOIN to bench GPUGPU performance instead of Folding @ Home? Um, okay.
I agree. I know Tom's spends a lot of time benchmarking, but Folding@home is something that is a bit more common. I would love to see F@H in some articles.
BTW, I appreciate all the work you guys do.
Where's the 5760x1080? In the area where I come from, 3x 1920x1080p 22" monitor cost around the same or even less than a single 2560x1440/1600 27" monitor so this is a more likely configuration among gamers.
The 5760x1080 resolution will also push the GPU's harder than a 2560x1440/1600 could so why limit the resolution there?
How far back in GPU generations are you going to test, if at all? I saw the power consumption charts and could only see GTX 500, 600 and Radeon 6000, 7000 series. I have an EVGA GTX 480 SC for two years and do like to know how it compares to the newer series of GPUs. Much appreciated.
Yup no surprise here typical Nvidia benchmark suite fuck sakes.
So what would YOU like to see used then? If they were trying to push Nvidia wouldn't Hawx 2 be in the suite?
For Starcraft II you say "This game doesn't stress the CPU, and is thus well-suited for GPU benchmarking." Looks like you got CPU and GPU mixed up there.
For Starcraft II you say "This game doesn't stress the CPU, and is thus well-suited for GPU benchmarking." Looks like you got CPU and GPU mixed up there.
More than likely, it's the sequence the German team picked that is less CPU-bound than other StarCraft II tests we've used.
Still with the bar charts? Would *love* to see scatter plots with price/score on the axes... So much more useful in picking out a card.
I would love something like this as well, but as Tom's has a world wide market and the prices vary so greatly from place to place such charts become impractical.
What I miss about the old charts is that you could easily compare old GPUs to new ones. When upgrading I think people like to have a reference of what they currently have compared to what they are looking at getting. To throw in a few old cards like the 8600GT, 8/9800GT in the mix may help people feel more secure in their purchases, and have a better feel for what they are getting because they can relate the new card to their old one. Not saying Tom's needs to put in every single card from every generation, but a card or 2 from each generation from the last 6 years or so would be nice.
Another thing I would like is something like Anandtech's bench where when you select your cards you get bar graphs instead of raw numbers in a chart. It is just easier to visually see a 1/3rd difference in performance on a line graph rather than in raw numbers.
I would have preferred to see all the benchmarks at 1920x1080 done with maxed out graphics settings.
I want to see how the base performance is in games with full detail, and save the heavy AA and resolutions for extreme.
while speaking of comparison chart complaints:
-Some cards (though not all) show up as their name, others have a picture of the card with a sales link, and others give very little information at all. It would be nice to make this consistent so that at the top of each column we could see the pic (if available), the name of the card, and then a sales link (hey, you have to pay of the site somehow)
-organize the comparison charts a little. On the comparison page it just throws all the charts together with no apparent rhyme or reason. It would be nice to have groupings such as physical considerations (temps, noise, and power usage), gpgpu benchmarks, and game benchmarks sorted either by game.
-Could we add physical dimensions? It would be helpful to some to know how long a card is, and now many slots it takes.
-lastly, under "02 - Unigine Heaven DirectX 11 [B] Performance 1920x1080, 4xAA, AFx16 Shaders medium Tessellation normal" the "score in" has a blank spot to enter in text (pretty sure it should say 'FPS' here) followed by a broken "Go" link.
None of these changes should be all that hard to make, and would make the charts much simpler to use when comparing specific cards.
Glad to see Bitcoin Mining being included now.
I would have preferred to see all the benchmarks at 1920x1080 done with maxed out graphics settings.I want to see how the base performance is in games with full detail, and save the heavy AA and resolutions for extreme.
This. I, along with probably a large chunk of your readers, are probably more interested in performance at maxed out settings at 1080p than skipping from mid-range settings at 1080p to maxed out settings at higher resolutions.
So what would YOU like to see used then? If they were trying to push Nvidia wouldn't Hawx 2 be in the suite?
Now that would just be to obvious.
Thanks for your work Guys,reading trough since long time and each rig has been built based upon your tests!
Gabriel
"What would you rather listen to, a low hum or a high-pitched whine at the same loudness?"
Lol. Good question. My tinnitus gives me a couple of the lows and several highs (but, just for fun and to keep me noticing them, the pitch and loudness vary). To be honest, I'd rather not hear any of them! ;O)