Skip to main content

How We Test Graphics Cards

Performance And Summary

Performance Measurements

As discussed, the only way to generate objective measurements is with a warmed-up card in a closed case, replicating the environmental conditions and clock rates gamers in the real world experience. Anything else is overly optimistic and subject to variance as test subjects heat up. We talk more about performance testing in PresentMon: Performance in DirectX, OpenGL, and Vulkan.

In addition to tools like FCAT, we have a tool we developed based on PresentMon. To save time, it allows us to store commonly-used games with corresponding profiles that only need to be created once, but can be edited any time. The profiles control PresentMon's many parameters when a stored application is detected.

We also have a configurable hot-key system for PresentMon, which makes it possible to start a test, control recording time and preferences, and manually exit PresentMon, all without leaving the game. A friendly voice pipes up to let us know when a desired action is successful, or whether it failed. In principle, the software works a lot like AMD's OCAT.

All the recorded data amounts to a huge pile of binary information that needs to be processed. To do this, we use another in-house tool that combines log files and performs calculations that'd be nearly impossible to manage through Excel.

The third generation of our log file interpreter smokes through these calculations almost instantly, and even allows us to build new analysis methods when they're necessary.

Conclusion

While our measurements are easier to collect, more applicable, and completely objective, it was the close cooperation with Lian Li that helped move this project forward. The closed case gave us the platform we've been needing, and that readers have been asking for, to facilitate more realistic reviews.

What use are benchmark results from a card fed fresh air when it behaves differently in the confines of a much warmer case? As an added benefit, measurements collected from inside a closed case help separate the wheat from the chaff; they help uncover the weaknesses we've seen poorly-designed graphics cards suffer from in the past. In the end, this all serves to help our readers make more informed buying decisions.


MORE: Best Graphics Cards


MORE: Desktop GPU Performance Hierarchy Table

  • DookieDraws
    Now that looks like a lab! Interesting read, Igor.
    Reply
  • kwc813
    I like the article. That said, will they make THAT BENCH (or offshoot therof!) to sell? I've seen most out there, used more than a few and am tired of customizing. This would allow one to customize everything else and leave the platform just as it sits. Nice job!
    Reply
  • zthomas
    love the glass box.. that system is high art.. the pics made my computer pic file.. to be posted elsewheres..
    Reply
  • edlight
    I remember when there were comparisons of video quality and enhancements between ATI & nvidia, and results of an "HQV" DVD quality test for multiple cards. I've seen no mention that AMD, in their drivers since 7.5.1, have cut off detailed settings for video - you can't combine high sharpening and dynamic contrast, for instance. And they even now don't let you control the overall brightness, contrast, and gamma. They refer you to an MS thing in Win 7 where you set the brightness on your monitor. Well, ever monitor I've had set to 100% brightness still needed an adjustment at the video card to bring out the shadow detail.
    The enhancements are great - I run my R7 240 on the old drivers at 100% sharpening and dynamic contrast on, affect web video, etc. DVD's in Media Player Classic look like something much better, and 1080 looks 3D. It only works on certain players, though. It works for Flash mp4-AVC. I've always wondered if it would work on everything in Linux?
    I see that the cheapest nvidia with the latest enhancements is the GTX 1050. I'm wondering if it can sharpen the new YouTube? I have to use a Firefox plugin to force YouTube back to Flash to get the sharpening.
    I wish reviewers would get into that stuff, and for each card.
    I'd love it if reviewers of video cards would attend to these things.
    Reply
  • Lucky_SLS
    Looking forward to the updated performance stats Igor. nicely done BTW
    Reply
  • FormatC
    19288871 said:
    I like the article. That said, will they make THAT BENCH (or offshoot therof!) to sell? I've seen most out there, used more than a few and am tired of customizing. This would allow one to customize everything else and leave the platform just as it sits. Nice job!

    I dicussed it with Lian Li and they will sell the core version of the T70 (without the cover) and an optional upgrade kit with all other parts to close it. This can help to minimize the financial risk for Lian Li and the costs on the customers side. Everybody can buy what he really wish and not verybody needs the full program. This closed "Real World" table was my idea and I hope, the industry and also reviewers will like it too.

    The lab here was built over the years, but I was every time not really satisfied with all my benchtables. The power consumption thing I started in 2013, the infrared measurement in 2014/15 - long time before other sites copied this. The audio lab was built with the help of a good friend to realize the room-in-room concept. The location is nearly perfect and I spent a lot of time and money to finish it. I'm an audio-freak and measured in Germany tons of speakers and headsets too. This was really successful because we have only HiFi magazines with a lot of phrases for the so-called Golden Ears or mostly very flat and useless reviews on websites without any measuring. I tried to get the right balance between theory and real life, combined with understandable data and conclusions.

    And who I am? I moved to media ( + product development/consultations) in 2012 and was working the half of my life in the German industry (the last position as lead programmer and for quality control). That also means, that I'm significant over 50 and also know the basics of production processes and a lot of "secrets" behind the scene. For reviews it is not only interesting, how a product performs and where are possible issues - it is every time very fascinating and exciting to find also solutions or workarounds (like EVGAs thermal pad mod) and to communicate with the manufacturers. I used and I'm using a lot of time to visit factories and headquarters in Asia to get even more contacs and sources.

    I worked in the last years mostly for Tom's Hardware Germany - but I'm really happy, that we found new translators to bring my content also to the US/UK site. So I'm here and you must endure this now ;)

    have to use a Firefox plugin to force YouTube back to Flash to get the sharpening.
    At first - thx for your suggestions. I will think about it. But especially Flash is a dead horse and it is not worth to spend more time for it. HTML5 is a better standard and YT is moving all content step by step. Maybe, we can make sometime a video special to take a closer look at all this problems. For HTPC builds it is not uninteresting :)
    Reply
  • zifn4b
    Dude! I use that wallpaper too. Excellent choice!
    Reply
  • Olle P
    Great idea! ... but:
    Air cooling isn't desirable for a number of reasons. First, a large tower-style heat sink would block too much of the IR camera's view.
    Possibly.
    Second, in a very small case like this, the cooler would dissipate so much waste heat that mainstream graphics cards with lower TDPs would be thermally overpowered, affecting the measurement results.
    I'd think the opposite way:
    * The heat generated by the CPU will be evacuated by the red fan. Shouldn't affect the graphics card one bit.
    * Some of the heat generated by the (high TDP) graphics card will impede the CPU cooling, which can affect CPU performance and thereby influence the test results. This is a viable part of real use and therefore a good thing to have in the test! It indirectly says if the graphics card will be more or less likely to require water cooling for the CPU to get the most performance out of it. (The main reason for having graphics coolers that blow as much of the heated air as possible directly out through the rear rather than just heating the case interior.)
    Reply
  • Unolocogringo
    Hi Igor,
    Just wanted to say that I really enjoy the depth and precision of your reviews.
    Ive been a member of Toms Hardware since 96, and have watched the progression of reviews evolve over the years.
    Your latest addition should make your reviews much more in line with consumer experience, and I applaud you for your effort and insight.
    Thanks
    Rick
    Reply
  • junkeymonkey
    why a 850w PSU ?? would it not be better to use a 550w seeing how there recommended here in the forums for a nice gaming rig ??

    if asked here at toms that 850w is way too much overkill .
    Reply