Sign in with
Sign up | Sign in

Test System And Benchmark Selection

Micro-Stuttering And GPU Scaling In CrossFire And SLI
By , Greg Ryder

Setup

Test System
CPU
Intel Core i5-2500K @ 5 GHz to verify GPU scaling
Cooling
Prolimatech SuperMega + Noiseblocker Multiframe PWM
Mainboard
Gigabyte Z68X-UD7


RAM
8 GB Kingston HyperX 1600 "Genesis"









System SSD
256 GB Samsung 470 Series
Power Supply
Cougar GX 105080 PLUS Gold
Total1050 W
Combined Power 3.3V/5V160 W
Combined Power 12V1008 W
Efficiency93 %

OS
Windows 7 x64 Ultimate
Test Equipment
Current and Power Meter
Energy Logger 4000 (Conrad Electronic)

• long-term measurements
• monitoring
• power measurements up to 1.2 KW
Voltcraft SBC-500 (Conrad Electronic)

• precise measurements down to the milliwatt range
• power measurements up to 500 watts
Sound Level Meter
Voltcraft SL-400 (Conrad Electronic)

• sound level measurements
• long-term recording
• monitoring
Syntethic Benchmarks
3DMark 11
  • Grahpics Tests 1-4
Unigine Sanctuary v.2.4
  • Dynamic lights
  • HDR rendering
  • Parallax occlusion mapping
  • Ambient occlusion mapping
  • Translucence
  • Volumetric light and fog
  • Particle systems
  • Postprocessing
  • Interactive experience with fly-through mode
Gaming Benchmarks
DirectX 11Metro 2033
Aliens Vs. Predator
DirectX 10Call of Juarez
S.T.A.L.K.E.R.: Call of Pripyat
DirectX 9Mafia II


Performance Categories: Gamer and Enthusiast

Although we cover a wide range of settings and resolutions in our launch coverage of various products, and will continue to do so, this piece includes two more generally-applicable performance categories that also take the acquisition cost into consideration. Our Gamer category is tested at 1680x1050, and our Enthusiast category is benchmarked at 1920x1080. We also base the image quality settings on both classifications.

Performance Index and Calculation Method

Instead of using cumulative frame rates to rank a graphics card, this review ranks cards in those same Gamer and Enthusiast classes. A graphics card that may be ranked rather low in the Enthusiast column might be completely usable in the Gamer category. Cumulative frame rates do not shed light on performance under specific conditions. Thus, we're replacing that old system with more transparent rankings.

So, how do we obtain these values? We take one representative from each GPU manufacturer, both for mid-range cards (AMD Radeon HD 6870 and Nvidia GeForce GTX 560 Ti) and high-end cards (AMD Radeon HD 6970, Nvidia GeForce GTX 570), and assess their behavior in 15 games. Then, we compute an average for each of the two card groups, creating two fictional 'Raforce Gedeon' cards. We use these two models as a reference for quality settings in each benchmark test and dub these synthetic frame rates the 100% level for the Gamer and Enthusiast categories, respectively, for each benchmark. The sum of all performance percentages, divided by the number of tests, yields two indexes. As the 'Raforce Gedeon' card is manufacturer-agnostic, advantages held by specific cards are eliminated from the get-go. This might sound complicated, but it really isn't. So long as you test using typical game settings, the rankings are a good indication of the price/performance ratio for each graphics card.

Power Draw and Temperature Measurement

We fully-loaded our CPU with Prime95 running threads at low priority in the background. Then, we measured the power draw of an entry-level graphics card with well-known power characteristics (first at idle, and then with FurMark running). Subtracting the card’s idle and load power from the total system power yielded, in each case, the total system power draw without a graphics card. These two values nearly matched, which allowed us to simply subtract 131 watts from all subsequent power measurements in order to obtain the power draw of the graphics card by itself. Of course, we also analyze the total power draw as a sanity check.

Display all 108 comments.
Top Comments
  • 28 Hide
    1kbuild , August 22, 2011 5:40 AM
    What happens with Vsync turned on?
  • 13 Hide
    shoelessinsight , August 22, 2011 7:55 AM
    What is performance like using other load-balancing methods, like the split frame rendering that SLI originally used, or ATI's Scissor mode? Are these modes still available to those that choose them?

    Obviously, they won't reach frame rates as high as those attained through AFR, but if the frame rate loss is small enough, those modes might still be justifiable if they eliminate micro-stuttering altogether.

    I'd be curious if these alternate methods could justify the cost of an additional card through added performance without coming with the drawback of micro-stuttering.
  • 11 Hide
    compton , August 22, 2011 5:24 AM
    Well, I'm a little surprised that three cards in Crossfire seem to eliminate visible microstuttering -- I would have guessed that triple cards would increase stuttering. But it also seems like there must be other factors at work. Unfortunately, there really isn't a good way to test for other factor -- if you even could know what to test for. In some circumstances, it seems like my monitor is causing some issues. If I play a game (lets use Fallout New Vegas for example) at a Synced 60FPS, you can look at FPS and it never deviates. It only uses 1/3 of my GPU cycles. But on one monitor, at the same resolution, it micro stutters. On another monitor, it looks perfectly fine. I thought it was some lag variance -- but then I've been told lag is always constant, that the reason lag varies in monitor testing is improper test methods. What ever the reason, it's actually really annoying. And I'm not anything approaching a competitive FPS player. Thanks for helping to track this issue down.



Other Comments
  • 9 Hide
    thorkle , August 22, 2011 5:16 AM
    This is a very interesting article, I have wondered about this issue myself many times in the past. I was always curious why I would see strange lag like anomalies while still achieving perfect frame rate. Bravo Toms
  • 11 Hide
    compton , August 22, 2011 5:24 AM
    Well, I'm a little surprised that three cards in Crossfire seem to eliminate visible microstuttering -- I would have guessed that triple cards would increase stuttering. But it also seems like there must be other factors at work. Unfortunately, there really isn't a good way to test for other factor -- if you even could know what to test for. In some circumstances, it seems like my monitor is causing some issues. If I play a game (lets use Fallout New Vegas for example) at a Synced 60FPS, you can look at FPS and it never deviates. It only uses 1/3 of my GPU cycles. But on one monitor, at the same resolution, it micro stutters. On another monitor, it looks perfectly fine. I thought it was some lag variance -- but then I've been told lag is always constant, that the reason lag varies in monitor testing is improper test methods. What ever the reason, it's actually really annoying. And I'm not anything approaching a competitive FPS player. Thanks for helping to track this issue down.



  • 28 Hide
    1kbuild , August 22, 2011 5:40 AM
    What happens with Vsync turned on?
  • 6 Hide
    pirateboy , August 22, 2011 7:00 AM
    why wasn't hybrid crossfire mentioned in this article?
  • -9 Hide
    bombat1994 , August 22, 2011 7:20 AM
    i think the MSI Radeon 6870 Hawk is the best graphics card on the market right now in terms of heat, power, performance and price
  • 8 Hide
    tmk221 , August 22, 2011 7:47 AM
    in most games you can limit maximum frame rates. So maybe if you limit max FR to let say 10 to 15 above min FR then the Micro-Stuttering effect would be gone ? anyone tried this? please share
  • 13 Hide
    shoelessinsight , August 22, 2011 7:55 AM
    What is performance like using other load-balancing methods, like the split frame rendering that SLI originally used, or ATI's Scissor mode? Are these modes still available to those that choose them?

    Obviously, they won't reach frame rates as high as those attained through AFR, but if the frame rate loss is small enough, those modes might still be justifiable if they eliminate micro-stuttering altogether.

    I'd be curious if these alternate methods could justify the cost of an additional card through added performance without coming with the drawback of micro-stuttering.
  • 7 Hide
    boletus , August 22, 2011 8:11 AM
    Regarding the decreased stuttering with 3 or 4 cards: could this be a similar effect to superimposing sine waves? Two waves a half cycle apart show visible peaks and valleys, while three waves at evenly staggered cycles form a much smoother band (on a graph or a scope).
  • -1 Hide
    haplo602 , August 22, 2011 8:18 AM
    I ma confused ... you are using 2 identical cards, so the frame rendering times as show on the metro 2033 second would be THE SAME on a single card as on a dual card configuration. the only difference is when each card starts to render right ?
  • 5 Hide
    cmcghee358 , August 22, 2011 8:18 AM
    Good question 1kbuild
  • 0 Hide
    Haserath , August 22, 2011 8:18 AM
    Could a frame be analyzed and split into certain pieces for each GPU to process? It would make each frame much more consistent if the load for each GPU was just about equal per frame even if there was overhead for splitting the load up and then having to transfer it to the first cards buffer.
  • -3 Hide
    damric , August 22, 2011 8:47 AM
    Disable ULPS you noobs.
  • -2 Hide
    SpadeM , August 22, 2011 8:50 AM
    I guess buying 3dfx was the way to go for Nvidia. If all midrange cards are equipped with only one link then I assume that we won't get to see exotic combinations of 3, 4 cards in the next SBM.
  • -3 Hide
    Samy0806 , August 22, 2011 9:38 AM
    Nice article, it was very usefull. BUT why Three-Way, and Quad SLI configurations aren't benchmarked ?
    AND i saw that Lucidlogix makes things worse on Radeon HD 6870 X2, what about the Lucidlocix Virtu, integrated in many motherboards, does it affects performance of your graphics card, and if it does how?
  • -2 Hide
    silverblue , August 22, 2011 10:55 AM
    Page 9, first Ungine Sanctuary set of results, the 590 is displayed using AMD Crossfire colours.
  • 2 Hide
    RazberyBandit , August 22, 2011 11:08 AM
    One reason single-GPU Nvidia boards may trail AMD boards in Mafia II is because Mafia II supports PhysX. With PhysX enabled, a single board can sometimes struggle.

    Based on the summary conclusion, does this mean Tom's has firmly gone back to recommending that users purchase "the most powerful single-GPU board you can afford" again?
  • 0 Hide
    nforce4max , August 22, 2011 11:34 AM
    shoelessinsightWhat is performance like using other load-balancing methods, like the split frame rendering that SLI originally used, or ATI's Scissor mode? Are these modes still available to those that choose them?Obviously, they won't reach frame rates as high as those attained through AFR, but if the frame rate loss is small enough, those modes might still be justifiable if they eliminate micro-stuttering altogether.I'd be curious if these alternate methods could justify the cost of an additional card through added performance without coming with the drawback of micro-stuttering.


    SFR was really good back in the day and it was enjoyable as well stable. Also it scaled well for two way setups but Nvidia killed it off because of quad and tri was becoming popular back in 08. ATI has other rendering modes as well such as tile but they went the afr rout as well. The best days of sli and crossfire are over but one can still try to optimize their way out of some micro shuttering.
  • 4 Hide
    BrightCandle , August 22, 2011 12:30 PM
    One of the members of XtremeSystems has come up with a program that analyses the amount of variance from the average framerate from a fraps frame time file. Have a look at http://www.xtremesystems.org/forums/showthread.php?258433-Microstutter-in-latest-gen-cards-examples-included&highlight=microstutter for a link and some details on how to use it and results.

    Toms could adopt this tool and use it to show the amount of MicroStutter along with their benchmark results. Many other sites like to show minimum fps in their graphs and I think showing the bottom 5% of frame times would be another way to show this problem up and compare the cards in your reviews.
  • -7 Hide
    greenrider02 , August 22, 2011 1:10 PM
    A great article, but of course a few stickler points:
    -It's not about Bulldozer
    -I would like to read some more detailed observations of problems you were seeing with crashing and driver issues
    -A suggestion in the conclusion about what YOU would do with your own setups, as many of us want to know what the pros think
    -It's not about Bulldozer
Display more comments
React To This Article