Sign in with
Sign up | Sign in

Graphics Card Myth Busting: How We Tested

The Myths Of Graphics Card Performance: Debunked, Part 1
By

Two Systems; Two Purposes

All of today's tests are performed on two separate rigs. One plays host to an older Intel Core i7-950, and another based on Intel's Core i7-4770K.

Test System 1
Enclosure
Corsair Obsidian Series 800D, Full Tower Case
CPU
Intel Core i7-950 (Bloomfield), Overclocked to 3.6 GHz, Hyper-Threading and power-saving features disabled
CPU Cooler
CoolIT Systems ACO-R120 ALC, Tuniq TX-4 TIM, Scythe GentleTyphoon 1850 RPM radiator fan
Motherboard
Asus Rampage III Formula
Intel LGA 1366, Intel X58 Chipset, BIOS: 903
Memory
Corsair CMX6GX3M3A1600C9, 3 x 2 GB, 1600 MT/s, CL 9
Graphic Cards
AMD Radeon R9 290X 4 GB (Press Board)
Nvidia GeForce GTX 690 4 GB
(Retail Board)
Nvidia GeForce GTX Titan 6 GB (Press Board)
Hard Drive
Samsung 840 Pro, 128GB SSD, SATA 6 Gb/s
Power Supply Unit
Corsair AX850, 850 W
Networking
Cisco-Linksys WMP600N (Ralink RT286) 
Audio Card
Asus Xonar Essence STX
Software and Drivers
Operating system
Windows 7 Enterprise x64, Aero disabled (see note below)
Windows 8.1 Pro x64 (for reference only)
DirectX
DirectX 11
Graphic Drivers
AMD Catalyst 13.11 Beta 9.5; Nvidia GeForce 331.82 WHQL
Test System 2
Enclosure
Cooler Master HAF XB, Desktop/Test bench hybrid format
CPU
Intel Core i7-4770k (Haswell), Overclocked to 4.6 GHz, Hyper-Threading and power-saving features disabled
CPU CoolerXigmatek Aegir SD128264, Xigmatek TIM, Xigmatek 120 mm fan
Motherboard
ASRock Extreme6/ac
Intel LGA 1150, Intel Z87 Chipset, BIOS: 2.20
Memory
G.Skill F3-2133C9D-8GAB, 2 x 4 GB, 2133 MT/s, CL 9
Graphic Cards
AMD Radeon R9 290X 4 GB (Press Board)
Nvidia GeForce GTX 690 4 GB
(Retail Board)
Nvidia GeForce GTX Titan 6 GB (Press Board)
Hard Drive
Samsung 840 Pro, 128GB SSD, SATA 6 Gb/s
Power Supply Unit
Cooler Master V1000, 1000 W
Networking
On-board 802.11ac mini-PCIe Wi-Fi card
Audio Card
On-board Realtek ALC1150
Software and Drivers
Operating system
Windows 8.1 Pro x64
DirectX
DirectX 11
Graphic Drivers
AMD Catalyst 13.11 Beta 9.5; Nvidia GeForce 332.21 WHQL

The first test system needed to facilitate repeatable results in a real-world environment. So, I set up a relatively old, but still very capable LGA 1366-based machine in a full-tower enclosure.

Test system number two needed to fulfill more specific requirements:

  1. Support PCIe 3.0 with a limited number of lanes (an LGA 1150-equipped board with a Haswell-based CPU, which only offers 16 lanes)
  2. Do not employ a PLX bridge chip
  3. Support three-way CrossFire in x8/x4/x4 or SLI in x8/x8 configurations

ASRock sent us its Z87 Extreme6/ac, which fit that description. We previously tested this board (minus the Wi-Fi module) in Five Z87 Motherboards Under $220, Reviewed, where it received our prestigious Smart Buy award. The sample we received was easy to set up, had no problem overclocking our Core i7-4770K sample to 4.6 GHz.

The board's UEFI gave me the option to set PCI Express transfer rates on a slot-by-slot basis, which enabled testing of PCIe Gen 1, 2 and 3 on the same motherboard. You will see the results of these tests in part 2 of this article.

Cooler Master supplied the second test system's chassis and power supply. The unconventional HAF XB enclosure, which also received Smart Buy honors in Cooler Master's HAF XB: Give Your LAN Party Box Breathing Room, proved comfortable to work with. It's very open, of course, so the components inside can get noisy if you don't have the right cooling setup. The case benefits from good airflow though, particularly if you hook up all of the optional fans.

The modular V1000 power supply allowed us to drive three high-end graphics cards, while containing cable clutter in a setting that was destined to get messy.

Comparing Test System 1 And 2

It's striking to see how similarly these systems perform once we get past their underlying architectures and focus on their frame rates. Here's a head-to-head between them in 3DMark Firestrike.

As you can see, the performance in graphics tests is essentially the same, even though the second machine has faster system memory (2133 versus 1800 MT/s, counter-balanced by Nehalem's triple-channel architecture compared to Haswell's two channels). Only in the host processor-dependent tests does the Core i7-4770K demonstrate an advantage.

The second system's main advantage is more overclocking headroom. Our Core i7-4770K sits at a stable 4.6 GHz on air, while the Core i7-950 can't exceed 4 GHz cooled by water. 

It's also worth noting that the first test system is benchmarked using Windows 7 x64 instead of Windows 8.1. There are three reasons for this:

  • First, the Windows desktop manager (Windows Aero or wdm.exe) uses a significant amount of graphics memory. At 2160p, it ties up an additional 200 MB in Windows 7 and 300 MB in Windows 8.1, on top of the 123 MB already reserved by Windows. This cannot be disabled without significant side effects in Windows 8.1, while it can be disabled in Windows 7 by switching to a basic theme. Four hundred megabytes is 20% of a 2 GB card's memory.
  • The memory usage in Windows 7 is consistent with a basic theme enabled. It is always 99 MB at 1080p and 123 MB at 2160p on a GeForce GTX 690. This makes for more repeatable tests. In contrast, the additional ~200 MB of memory used by Aero varies up and down by roughly 40 MB.
  • As of Nvidia's 331.82 WHQL driver, a bug exists affecting 2160p when Windows Aero is enabled. This only surfaces when Aero is enabled on a tiled 4K display, and it manifests itself as lowered GPU utilization during benchmarks (bouncing in the 60-80% range, instead of close to 100%), and a resulting drop in performance of 15% or so. Nvidia was notified of this.
Additional testing equipment
Screen photography
Canon EOS 400D
Canon EF 50 mm f/1.8 lens
1/400s, ISO 800, f/1.8-2.8
Sound Pressure Level monitor
ART SPL-8810, dB(A)/Low/Fast setting

Tearing and ghosting effects do not show up in regular screen shots or game videos; I used a fast camera to capture the actual on-screen image.

Case ambient temperature is measured with the Samsung 840 Pro integrated temperature sensor. Background ambient temperature was in the range of 20-22 °C (68-72 °F). Background sound pressure level for all noise tests was 33.7 dB(A), +/- 0.5 dB(A).

Benchmark Configuration
3D Games
The Elder Scrolls V: Skyrim
Version 1.9.32.0.8, Custom THG Benchmark, 25-Sec. HWiNFO64
Hitman: Absolution
Version 1.0.447.0, Built-in Benchmark, HWiNFO64
Total War: Rome 2
Patch 7, Built-in "Forest" Benchmark, HWiNFO64
BioShock Infinite
Patch 11, Version 1.0.1593882, Built-in Benchmark, HWiNFO64
Synthetic Benchmarks
Ungine Valley
Version 1.0, ExtremeHD Preset, HWiNFO64
3DMark Fire Strike [Extreme]Version 1.1

A variety of tools can be used for measuring graphics card memory use. We went with HWiNFO64, taking advantage of its maximum mark. The same results can be obtained through MSI Afterburner, EVGA Precision X, or simply the RivaTuner Statistics Server stand-alone.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 135 comments.
This thread is closed for comments
Top Comments
  • 26 Hide
    blackmagnum , February 10, 2014 1:08 AM
    Myth #123: Gamers are lonely boys in Mother's dark basement or attic...
  • 16 Hide
    cats_Paw , February 10, 2014 4:45 AM
    Awsometacular article.Not only its a new standard for GPU performance, but the Human Benchmark and audio test was really fun!Im normally very critisizing about toms articles becouse many times they feel a bit weak, but this one?10/10
  • 12 Hide
    Jaroslav Jandek , February 10, 2014 5:38 AM
    Quote:
    The info on V-Sync causing frame rate halving is out of date by about a decade. With multithreading the game can work on the next frame while the previous frame is waiting for V-Sync. Just look at BF3 with V-Sync on you get a continous range of FPS under 60 not just integer multiples. DirectX doesn't support triple buffering.
    The behavior of V-Sync is implementation-specific (GPU drivers/engine). By using render ahead, swap chains, Adaptive V-Sync, etc., you can avoid frame halving.

    DirectX DOES support TB by using DXGI_SWAP_CHAIN_DESC.BufferCount = 3; (or D3DPRESENT_PARAMETERS.BackBufferCount = 2; for DX9). It actually supports more than triple buffering - Direct3D 9Ex (Vista+'s WDDM) supports 30 buffers.
Other Comments
  • 5 Hide
    ingtar33 , February 10, 2014 12:43 AM
    awesome article, looking forward to the next half.
  • 26 Hide
    blackmagnum , February 10, 2014 1:08 AM
    Myth #123: Gamers are lonely boys in Mother's dark basement or attic...
  • 4 Hide
    AlexSmith96 , February 10, 2014 1:09 AM
    Great Article! I love you guys for coming up with such a nice idea.
  • 2 Hide
    hansrotec , February 10, 2014 1:09 AM
    with over clocking are you going to cover water cooling? it would seem disingenuous to dismiss overclocking based on a generating of cards designed to run up to maybe a speed if there is headroom and not include watercooling which reduces noise and temperature . my 7970 (pre ghz editon) is a whole different card water cooled vs air cooled. 1150 mhz without having to mess with the voltage on water with temps in 50c without the fans or pumps ever kicking up, where as on air that would be in the upper 70s lower 80s and really loud. on top of that tweeking memory incorrectly can lower frame rate
  • 6 Hide
    hansrotec , February 10, 2014 1:18 AM
    I thought my last comment might have seemed to negative, and i did not mean it in that light. I did enjoy the read, and look forward to more!
  • -1 Hide
    hansrotec , February 10, 2014 1:22 AM
    I thought my last comment might have seemed to negative, and i did not mean it in that light. I did enjoy the read, and look forward to more!
  • -1 Hide
    noobzilla771 , February 10, 2014 1:26 AM
    Nice article! I would like to know more about overclocking, specifically core clock and memory clock ratio. Does it matter to keep a certain ratio between the two or can I overclock either as much as I want? Thanks!
  • 5 Hide
    chimera201 , February 10, 2014 1:28 AM
    I can never win over input latency no matter what hardware i buy because of my shitty ISP
  • -1 Hide
    immanuel_aj , February 10, 2014 2:00 AM
    I'd just like to mention that the dB(A) scale is attempting to correct for perceived human hearing. While it is true that 20 dB is 10 times louder than 10 dB, but because of the way our ears work, it would seem that it is only twice as loud. At least, that's the way the A-weighting is supposed to work. Apparently there are a few kinks...
  • 0 Hide
    FunSurfer , February 10, 2014 3:35 AM
    On Page 3: "In the image below" should be "In the image above"
  • -1 Hide
    Formata , February 10, 2014 3:37 AM
    "Performance Envelope" = GeniusNice work Filippo
  • -1 Hide
    beetlejuicegr , February 10, 2014 4:19 AM
    I just want to mention that db is one thing, health of gpu over time is another. In many cases i have seen graphic cards going up to 90C before the default driver of ATI/Nvidia start to throttle down. i prefer a 50C-70C scenario
  • 16 Hide
    cats_Paw , February 10, 2014 4:45 AM
    Awsometacular article.Not only its a new standard for GPU performance, but the Human Benchmark and audio test was really fun!Im normally very critisizing about toms articles becouse many times they feel a bit weak, but this one?10/10
  • 0 Hide
    ubercake , February 10, 2014 5:00 AM
    What's up with Precision X? It seems like they would update it every couple of months and now there hasn't been an update since last June or July?Is EVGA getting out of the utility software business?
  • 8 Hide
    kzaske , February 10, 2014 5:01 AM
    Its' been a long time since Tom's Hardware had such a good article. Very informative and easy to read. Thank you!
  • -1 Hide
    ddpruitt , February 10, 2014 5:04 AM
    Very good article even though there are some technical errors. I look forward to seeing the second half! I would also be interesting in seeing some detailed comparisons of the same cards with different amounts and types of VRAM and case types on the overall impact of performance.
  • 12 Hide
    Jaroslav Jandek , February 10, 2014 5:38 AM
    Quote:
    The info on V-Sync causing frame rate halving is out of date by about a decade. With multithreading the game can work on the next frame while the previous frame is waiting for V-Sync. Just look at BF3 with V-Sync on you get a continous range of FPS under 60 not just integer multiples. DirectX doesn't support triple buffering.
    The behavior of V-Sync is implementation-specific (GPU drivers/engine). By using render ahead, swap chains, Adaptive V-Sync, etc., you can avoid frame halving.

    DirectX DOES support TB by using DXGI_SWAP_CHAIN_DESC.BufferCount = 3; (or D3DPRESENT_PARAMETERS.BackBufferCount = 2; for DX9). It actually supports more than triple buffering - Direct3D 9Ex (Vista+'s WDDM) supports 30 buffers.
  • 8 Hide
    Adroid , February 10, 2014 5:55 AM
    I would love to see a Tom's article on debunking the 2GB vs 4GB graphic card race. For instance, people spam the Tom's forum daily giving advice to buy the 4GB GTX 770 over the 2GB. Truth is, the 4 GB costs 50$ more and offers NO benefit over the 2GB. Even worse, I see people buying/suggesting the 4GB 760 over a 2GB 770 (which runs only 30$ more and is worth every penny). I am also curious about the 4GB 770 sli scenario. For everything I have seen, even in Sli the 4GB offers no real-world benefit (with the exclusion of MAYBE a few frames per second higher at 3 monitor scenarios, but the rates are unplayable regardless so the gain is negligible). The other myth is that the 4GB 770 is more "future proof". Give me a break. GPU and future proof do not belong in the same sentence. Further, if they were going to be "future proof" they would be "now proof". There are games that are plenty demanding to show the advantage of 2gb vs 4gb - and they simply don't. It's tiring seeing people giving shoddy advice all over the net. I wish a reputable website (Tom's) would settle it once and for all. In my opinion, the extra 2 GB of RAM isn't going to make a tangible difference unless the GPU architecture changes...
  • 0 Hide
    ubercake , February 10, 2014 5:55 AM
    DisplayLag.com lists 120Hz and 240Hz HDTVs amongst the monitors, but the maximum input speed for the HDTVs' inputs equate to 60fps? Or am I missing something?If I buy a 240Hz refresh TV, that's output. It processes the 60Hz signal to transform it to a 240Hz output (usually through some form of frame duplication) to minimize motion blur. Does this displayLag.com site mentioned in the article compare apples to oranges by listing HDTVs with monitors as if they operate the same way or am I way off here?
Display more comments