GFXBench 3.0: A Fresh Look At Mobile Benchmarking

Special Test Results: Battery Life And Performance

With devices like Nvidia's Tegra K1 announced, there are closer-than-ever ties between desktop and mobile architectures, in addition to the APIs they support. As a result, the games you can (and will be able to) play on your smartphone and tablet are becoming more advanced. Developers continue creating new IP for gamers. Moreover, we have high hopes that popular PC and console titles will start making their way over as well. Gamers naturally expect more from their devices, and this is taking a toll on battery life. The next frontier of mobile advancement, we believe, has to come from the power side.

Now, typically we try to run contrived scenarios to evaluate the battery life of the devices we review. This isn't easy. Particularly with cross-platform comparisons, comparable workloads that take a battery from 100 to 0 percent are not common, particularly when it comes to gaming.

Needless to say, GFXBench's battery test is something we're evaluating more closely. It measures battery life and performance stability by logging frame and battery discharge rate as the T-Rex test loops at least 30 times. The results are given in two scores: estimated battery life in minutes, and the number of frames rendered on the slowest test run (to gauge if a device is throttling). Both tests are run at the device’s 50% brightness level in the free Community edition, while the paid Corporate edition can be set to 0%, 25%, 75%, 100%, or whatever the device’s native slider is set to. We very specifically calibrate all units to 200 nits before testing.

Battery Lifetime

Let’s see what battery performance we can eke out of these gadgets.

The Nexus 7 and Oppo N1 actually sport very similar hardware, the former equipped with Qualcomm's S4 Pro at up to 1.5 GHz and the latter with a Snapdragon 600 at up to 1.7 GHz. Google's tablet sports a larger 1920x1200 display, but also includes a beefy 3950 mAh battery that conveys an advantage over the N1's 3610 mAh power source.

If we took these results on their own, it'd be surprising to see the Galaxy Note 10.1” 2014 Edition take third place, given its 8220 mAh battery. Bear in mind, though, that Samsung's larger tablet employs a 2560x1600 panel. Moreover, we just saw the Note's Mali-T628MP6 GPU put a big emphasis on high-quality graphics, which has an impact on battery life as well.

Placing fourth, EVGA's Tegra Note 7 falls only 11 minutes short of the Galaxy Note 10.1”, which is pretty impressive when you consider its substantially smaller 4100 mAh battery.

Google's Nexus 5 does really well in our performance tests thanks to its quad-core processor complex and Adreno 330 graphics engine. However, great performance and a relatively small 2300 mAh battery don't facilitate long gaming marathons on the road.

Of course, that's not as bad as the Meizu's MX3, which has a slightly larger battery, a slower PowerVR GPU, but still ends up second-to-last.

And then there's Apple's iPhone 5s. A potent A7 SoC and diminutive 1560 mAh would seem to disappoint. But there's an important point to make here. GFXBench 3.0 isn't running all of these devices at a constant frame rate, so their performance directly affects battery life. In the case of Apple's flagship smartphone, it turns in the top frame rate recorded in GFXBench's online database (37 FPS) through the T-Rex benchmark, so it's actually working harder and draining faster. If your 3D application of choice is already capped at 60 FPS, the iPhone might not need to go all-out, and its battery would therefore last longer.

Battery Performance

Of course, Kishonti is running the battery test like this deliberately. We want to know the slowest iteration's values specifically to identify whether a device is running too hot, and therefore throttling back performance as the benchmark progresses. Holding a steady frame rate would mess with that investigation.

Actually, what I'd really like to see from GFXBench 3.0 would be a frame rate for each of the battery test's runs, which would let us determine when, exactly, a throttling device drops its performance. For now, the best we can manage is a comparison between our initial T-Rex test and the 30-iteration version.

A quick comparison between this chart and the on-screen test from page three shows that Apple's first-place performance holds steady; the iPhone 5s doesn't throttle and indeed delivers a consistent frame rate, even after 30 runs.

Similarly, Google's Nexus 5 yields a performance figure that closely mirrors the single-run pass of T-Rex we reported on page three. We see that better battery life compared to the iPhone 5s costs you raw performance from the Snapdragon 800 SoC, which just isn't as fast. Still, at least you can count on the phone serving up a consistent experience.

The same is not true for EVGA's Tegra Note 7; the Tegra 4-powered tablet's first run registers 30 FPS. By the time we hit the thirtieth, performance is down to 21 FPS. Although it shows well on the battery life chart, it'd make sense that a tablet would last longer when it gets throttled back.

Google's Nexus 7, Oppo's N1, and Meizu's MX3 are all over the map when it comes to battery life, but at least they're able to maintain performance through 30 iterations of T-Rex. 

Meanwhile, Samsung's Galaxy 10.1" appears to render fewer frames in its slowest run. Our notes show that one run was very consistent, while a second dropped 2 FPS at worst. The Exynos 5420-powered tablet is already one of the slowest in GFXBench's T-Rex module, so we're inclined to think there's some variance in play.

  • Cryio
    I was just about to write "why not WP", but then I remember WP games run on DirectX.
    Reply
  • panzerknacker
    Its cool u guys put so much effort into this but tbh most of the benchmark results seem to be completely random. Phones with faster SoC's performing slower and vice versa. I think there is no point at all benching a phone because 1. The benchmarking software is a POS and unreliable and 2. The phone OS's and apps are all complete POSs and act completely random in all kinda situations. I'd say just buy the phone with a fast SoC that looks the best to u and when it starts acting like a POS (which they all start doing in the end) buy a new one.
    Reply
  • Marcus Wandle
    You show those dumb nay sayers, Apple.
    Reply
  • umadbro
    What kind of bs is this? Force 720p on all devices and you'll see what happens to your precious 5s. Even my Zl murdered it.
    Reply
  • andreluizbarbieri
    Why No mention about MX3 and Note beat iphone 5s?
    Reply
  • jamsbong
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    Reply
  • rolli59
    Well I have a smart phone but that is so I can receive business emails on the go, I have a tablet because it is great for watching movies on the go. Do I want to find out if there are any faster devices to do those things, not really while what I got is sufficient. I leave all the heavy tasks to the computers.
    Reply
  • Durandul
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
    Reply
  • umadbro
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
    It does give the option to force some specific resolution. Don't know why this "review" didn't do it. That's what I've been trying to say from the start.
    Reply
  • umadbro
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
    It does give the option to force some specific resolution. Don't know why this "review" didn't do it. That's what I've been trying to say from the start.
    Reply