Challenging FPS: Testing SLI And CrossFire Using Video Capture

By Don Woligroski
published

What if the performance data you used for deciding which graphics card to buy was flawed? We're taking a deeper look at some of the problems with benchmarking multi-GPU configs using conventional tools. Nvidia's new FCAT suite helps us collect more info.

Results: Batman Arkham City

With results to compare in nine games, we're proceeding alphabetically. Batman Arkham Asylum is first in line.

First, we'll look at the minimum and average frame rates. When you see "Hardware FPS" in the charts, you're looking at the number of frames per second the graphics card is rendering. "Practical FPS" is the more accurate representation of what you'd actually see, with dropped frames and runts taken out of the equation.

In addition, we have a fifth data point, captured in Fraps on the CrossFire-based system, to use as a comparison point.

The difference between the hardware and practical frame rates on our GeForce GTX 660 Ti-based SLI setup is only about one frame per second. In comparison, the gap is more like five frames per second on the Radeon HD 7870-based CrossFire configuration.

Fraps would have us believe that the hardware FPS number is right, even though the practical frame rate is lower.

The frame rate over time chart shows us that the Radeon configuration's hardware FPS (the thin red line) and the Fraps result (the thin red line with dots) sometimes spike above the practical FPS (the thick red line). When it comes to the GeForce cards, both lines remain close together.

For reference, we're also including our frame time variance chart, generated from the data generated by FCAT. The GeForce and Radeon cards exhibit sub-10 ms results when we look at the 95th percentile.

Don Woligroski
198 Comments Comment from the forums
  • kajunchicken
    Hopefully someone besides Nvidia develops this technology. If no one does, Nvidia can charge whatever they want...
    Reply
  • cleeve
    kajunchickenHopefully someone besides Nvidia develops this technology. If no one does, Nvidia can charge whatever they want...
    FCAT isn't for end users, it's for review sites. The tech is supplied by hardware manufacturers, Nvidia just makes the scripts. They gave them to us for testing.
    Reply
  • cangelini
    kajunchickenHopefully someone besides Nvidia develops this technology. If no one does, Nvidia can charge whatever they want...And actually, it'd be nice to see someone like Beepa incorporate the overlay functionality, taking Nvidia out of the equation.
    Reply
  • cravin
    I wish there were an easy way to make my frame rates not dip and spike so much. A lot of times it can go up to like 120fps but then dipts down to 60 -70. It makes it look super choppy and ugly. I know I could limit it at 60 frames a second, but wouldn't that just be like vsync? Would that have input lag.
    Reply
  • DarkMantle
    Good review, but honestly I wouldnt use a tool touched by Nvidia to test AMD hardware, Nvidia has a track record of crippling the competition's hardware every chance they have. Also, i was checking prices in Newegg and to be honest the HD7870 is much cheaper than the GTX660Ti, why didn't you use 7870LE (Tahiti core) for this test? The price is much more closer.
    The problem i have with the hardware you picked for this reviews is that even though, RAW FPS are not the main idea behind the review, you are giving a Tool for every troll on the net to say AMD hardware or drivers are crap. The idea behind the review is good though.
    Reply
  • krneki_05
    Vsync would only cut out the frames above 60FPS, so you would still have the FPS drops. but 60 or 70 FPS is more then you need (unless you are using 120Hz monitor and you have super awesome eyes to see the difference between 60 and 120FPS, that some do). No, the choppy felling you have must be something else not the frames.
    Reply
  • BS Marketing
    Does it really matter? Over 60 FPS there will be screen tearing. So why is this sudden fuss? I guess nvidia marketing engine is in full flow. Only explanation is that Nvidia is really scared now, trying everything in there power to deceive people.
    Reply
  • rojodogg
    I enjoyed the artical and it was very informative, I look forward to more testing of other GPU-s in the future.
    Reply
  • bystander
    Great article.

    But as great as the review is, I feel one thing that review sites have dropped the ball on is the lack of v-sync comparisons. A lot of people play with v-sync, and while a 60hz monitor is going to limit what you can test, you could get a 120hz or 144hz monitor and see how they behave with v-sync on.

    And the toughest thing of all, is how can microstutter be more accurately quantified. Not counting the runt frames gives a more accurate representation of FPS, but does not quantify microstutter that may be happening as a result.

    It seems the more info we get, the more questions I have.
    Reply
  • rene13cross
    @DarkMantle, exactly my thinking. I don't want to sound like a paranoid goof who thinks everything is a conspiracy but a test suite created by Nvidia to test AMD hardware doesn't sound like a very trustworthy test. I'm not saying that the results here are all false but Nvidia has had a slight history in the past with attempting to present the competition in an unfair light.
    Reply