Introducing Our Reference System And Methodology For 2014
Our Mission: The Best Data Possible
Over the past year or so, you've seen us make some significant changes to many of our graphics card reviews. For example, whenever possible, we employ the Frame Capture Analysis Tool, dubbed FCAT and covered in depth in Challenging FPS: Testing SLI And CrossFire Using Video Capture. Our performance results are consequently more accurate than they've ever been.
Recently, we decided to dig deeper into power consumption as well, approaching it with the same precision, and you've likely seen the product of that in our more recent launch coverage. We don't want to just give you a rough estimate of what a given graphics card draws. No, the goal here is to set a new standard for power measurement. This is particularly important to us at a time when GPU vendors aren't just talking raw performance any more. They're putting an emphasis on efficiency, valuing cards able to achieve high frame rates without dissipating a ton of heat.
A new methodology, developed in concert with an industry partner, allows us to ask (and answer) questions that couldn't be easily addressed in the past. Just days ago, in Radeon R9 295X2 8 GB Review: Project Hydra Gets Liquid Cooling, we were able to compare the Radeon R9 295X2, HD 7990, and HD 6990, telling you how much power each card pulled over its PCI Express slot and eight-pin connectors. In the days to come, we'll publish a follow-up explaining why all of this matters so much, allowing you to interpret the outcome of our testing more easily.
The other purpose of today's story is to introduce our 2014 Graphics Card Charts. To account for advances in display technology, we're testing synthetics and real-world games at two resolutions now: FHD (1920x1080) and UHD (3840x2160). You'll see a second set of resolutions and results a little later; those will account for entry-level boards, APUs, and Intel's integrated graphics solutions.
Benchmarking On A New Reference System
Over the course of the last year, we've seen plenty of cases where an Intel Core i7-3770K, even one overclocked to 4.5 GHz, can become a bottleneck. Fortunately, at the resolutions we picked and the settings we're running at, limitations should be few. To further assure this, we also built up a new reference machine based on Intel's Core i7-4930K operating at 4 GHz, a very fast quad-channel memory kit, and Asus' Rampage IV Black Edition motherboard.
Freezing the Current State to Ensure Fair Comparisons
The evolution of gaming confounds us every year. It's easy enough to adopt new titles and retire old ones in our reviews. But it's a lot harder to collect a ton of data for these persistent charts and watch it age over the course of 12 months. We know it'd be impossible to use every single popular game for this project, particularly since we're doing multiple benchmark runs for each one. So, our team picked 10 AAA titles with fairly long sequences and available settings to test. The suite we ended up with purposely strikes a balance between graphics card vendors, too.
We’re using both modern and older games, which accordingly present a range of challenges for the graphics hardware. We deactivated automatic updates, effectively freezing their current state as of early 2014. And the copy of Windows we're using is also kept from phoning home for patches. This configuration is saved as an image, allowing us to reuse it over and over. Whenever it's necessary, we also use new drivers. Sometimes that necessitates applying them retroactively and re-testing cards, particularly when the performance of a card is affected in a big way.
