Sign in with
Sign up | Sign in

GFXBench 3.0: A Fresh Look At Mobile Benchmarking

GFXBench 3.0: A Fresh Look At Mobile Benchmarking
By

Over the past few years, Kishonti has become a leading name in mobile GPU benchmarking. The newly-released GFXBench 3.0 is comprised of nearly all new tests, including battery, render quality, and the first serious OpenGL ES 3.0 performance metric.

Graphics standards continue to evolve, and benchmarks are frequently updated to keep pace. In many ways, GFXBench 3.0 is a new beast, featuring the suite's first OpenGL ES 3.0-based performance test, measures of image quality, system impact, and battery performance.

Previous GFXBench versions were comprised of traditional “high-level” game-like scenarios along with more “low-level” tests designed to measure specific subsystems. Version 3.0 expands the software's scope, retaining only one sequence from previous builds: the OpenGL ES 2.0-level T-Rex HD from GFXBench v2.7. The old faithful Egypt HD test from v2.5 retires, which makes sense, since even modern entry-level graphics engines power through it with ease. Naturally, it's no longer challenging for mid-range and high-end SoCs. In its place is the far more demanding Manhattan test, utilizing OpenGL ES 3.0-specific complex lighting, particles, and, most important, deferred shading.

The Low Level performance benchmarks have also been improved over v2.7. An ALU test was added to calculate raw shader performance, while the new Alpha Blending test does the same for rendering multiple transparent objects on top of each other. In addition, there’s a new set of render quality tests that evaluate a device's fidelity by comparing a single rendered frame to a reference, scoring the outcome in peak signal-to-noise ratio. One version forces the shaders to run with high precision, and the other doesn't. A Driver Overhead test shows how heavily a CPU complex is affected by draw calls and state changes.

Finally, a brand new battery test was added, rendering the T-Rex HD in a loop at 50% screen brightness and logging frame rate as the test iterates at least 30 times. While Manhattan may get all the spotlight for emphasizing current-gen graphics capabilities, this is also a very important addition. It, along with the quality metric, gives us a way to compare performance to output and longevity. Device makers and SoC vendors who optimize for one vector are going to negatively affect the others.

Today we’ll give you a test-by-test analysis of GFXBench 3.0 using a selection of devices that run the gamut of modern SoCs.

Device
SoC
CPU Core
GPU Core
Memory
Display
Battery
Apple iPhone 5s
Apple A7
ARM v8 (dual-core) @ 1.3 GHz
Imagination Technologies PowerVR G6430 (four-cluster) @ 300 MHz
1 GB DDR3
4" IPS @ 1136x640 (326 PPI)
1560 mAh
EVGA Tegra Note 7
Nvidia Tegra 4 (T114)
ARM Cortex-A15 (quad-core) @ 1.8 GHz
ARM Cortex -A15 (single companion-core) @ 500 MHz
GeForce ULP (72-core) @ 672 MHz
1 GB DDR3
7" IPS @ 1280x800 (216 PPI)
4100 mAh
Google Nexus 5
Qualcomm Snapdragon 800 (8974-AA)
Qualcomm Krait 400 (quad-core) @ 2.3 GHzQualcomm Adreno 330 (quad-core) @ 450 MHz2 GB DDR34.94” IPS+ @ 1920x1080 (445 PPI)2300 mAh
Google Nexus 7
Qualcomm Snapdragon S4 Pro (APQ8064-1AA)Qualcomm Krait 300 (quad-core) @ 1.5 GHzQualcomm Adreno 320 (quad-core) @ 400 MHz2 GB DDR37.1” IPS @ 1920x1200 (323 PPI)3950 mAh
Meizu MX3
Samsung Exynos 5 Octa (5410)ARM Cortex-A15 (quad-core) @ 1.6 GHz
ARM Cortex-A7 (quad-core) @ 1.2 GHz
Imagination Technologies PowerVR SGX544MP3 (tri-core) @ 480 MHz2 GB DDR35.1” IPS @ 1800x1080 (412 PPI)2400 mAh (Li-Pro)
Oppo N1
Qualcomm Snapdragon 600 (APQ8064T)Qualcomm Krait 300 (quad-core) @ 1.7 GHzQualcomm Adreno 320 (quad-core) @ 400 MHz2 GB DDR35.9” IPS @ 1920x1080 (373 PPI)3610 mAh
Samsung Galaxy Note 10.1" 2014 Edition
Samsung Exynos 5 Octa (5420)ARM Cortex-A15 (quad-core) @ 1.9 GHz
ARM Cortex-A7 (quad-core) @ 1.2 GHz
ARM Mali-T628MP6 (hexa-core) @ 480-600 MHz3 GB DDR310.1” WQXGA TFT @ 2560x1600 (229 PPI)8220 mAh

Let’s start out with Manhattan, GFXBench’s newest High-Level test, and the first serious OpenGL ES 3.0 benchmark to hit the scene.

Display all 21 comments.
This thread is closed for comments
  • 0 Hide
    Cryio , February 21, 2014 1:30 AM
    I was just about to write "why not WP", but then I remember WP games run on DirectX.
  • -2 Hide
    panzerknacker , February 21, 2014 3:18 AM
    Its cool u guys put so much effort into this but tbh most of the benchmark results seem to be completely random. Phones with faster SoC's performing slower and vice versa. I think there is no point at all benching a phone because 1. The benchmarking software is a POS and unreliable and 2. The phone OS's and apps are all complete POSs and act completely random in all kinda situations. I'd say just buy the phone with a fast SoC that looks the best to u and when it starts acting like a POS (which they all start doing in the end) buy a new one.
  • -1 Hide
    Marcus Wandle , February 21, 2014 4:24 AM
    You show those dumb nay sayers, Apple.
  • -6 Hide
    umadbro , February 21, 2014 6:15 AM
    What kind of bs is this? Force 720p on all devices and you'll see what happens to your precious 5s. Even my Zl murdered it.
  • -2 Hide
    andreluizbarbieri , February 21, 2014 6:17 AM
    Why No mention about MX3 and Note beat iphone 5s?
  • 0 Hide
    jamsbong , February 21, 2014 8:58 AM
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
  • -1 Hide
    rolli59 , February 21, 2014 9:23 AM
    Well I have a smart phone but that is so I can receive business emails on the go, I have a tablet because it is great for watching movies on the go. Do I want to find out if there are any faster devices to do those things, not really while what I got is sufficient. I leave all the heavy tasks to the computers.
  • 2 Hide
    Durandul , February 21, 2014 10:38 AM
    Quote:
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
  • 2 Hide
    umadbro , February 21, 2014 11:34 AM
    Quote:
    Quote:
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
    It does give the option to force some specific resolution. Don't know why this "review" didn't do it. That's what I've been trying to say from the start.
  • 0 Hide
    umadbro , February 21, 2014 11:41 AM
    Quote:
    Quote:
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
    It does give the option to force some specific resolution. Don't know why this "review" didn't do it. That's what I've been trying to say from the start.
  • 3 Hide
    dragonsqrrl , February 21, 2014 2:52 PM
    Quote:
    Quote:
    Quote:
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
    It does give the option to force some specific resolution. Don't know why this "review" didn't do it. That's what I've been trying to say from the start.

    ... you guys realize that the off-screen tests render at 1080p, right? That's the whole point, to make direct performance comparisons regardless of a devices display resolution. It's also explained in the performance results.

    On a different note, I find it amazing how consistently and predictably the community on this site tries to discredit an objective review when the performance results favor an Apple device in any way. This isn't exactly breaking news for anyone who's familiar with SOC performance. Please try to set aside your childish biases and just accept the results for what they are. The A7 is a powerful SOC, get over it.
  • 0 Hide
    h2323 , February 21, 2014 10:25 PM
    Looks like imagination techs powervr and the radeon..I mean adreno own.
  • -1 Hide
    umadbro , February 22, 2014 12:29 AM
    Quote:
    Quote:
    Quote:
    Quote:
    The only relevant benchmarks are the first two because they are full-fletch 3D graphics, which is won by the most portable device; The iPhone. The rest of the benchies are just primitive 2D graphics which is irrelevant. Android devices won all those in flying colours.
    If those are the only two benchmarks relevant to you, then I wonder why you are using a phone and not a 3DS or something. But seriously, most of the other devices have more than a million more pixels then the iPhone, so this benchmark is not so telling. It was mentioned before, but it would be nice to test at a given resolution, although as suppose applications don't give you an option on the phone.
    It does give the option to force some specific resolution. Don't know why this "review" didn't do it. That's what I've been trying to say from the start.
    ... you guys realize that the off-screen tests render at 1080p, right? That's the whole point, to make direct performance comparisons regardless of a devices display resolution. It's also explained in the performance results.On a different note, I find it amazing how consistently and predictably the community on this site tries to discredit an objective review when the performance results favor an Apple device in any way. This isn't exactly breaking news for anyone who's familiar with SOC performance. Please try to set aside your childish biases and just accept the results for what they are. The A7 is a powerful SOC, get over it.
    I love people like you who claim to be some proffesinal "SoC experts" online while we the rest of us don't know nothing.A7 is a powerful SoC but the GPU is the same powervr as many other devices have. Apple tweaked the cpu cores only (at least going by the news).You talk about the off-screen tests which show exactly that the Android powered devices clearly pull back into the game with the A7, only one's the iPhone gets are the on screen tests which the iPhone runs at 720 and the androids run at 1080 - million(s) of more pixels to process then of course the load on the gpu is bigger therefor slower results. That's exactly why this review needs to force 720 on everything which you can easily do in the app itself on android.The two cores in an A7 are tweaked so much it keeps up with quad-core SD's etc, I got to give them that. But don't come rushing in telling that you know it all and everyone else are just dumb.
  • 0 Hide
    nebun , February 22, 2014 8:01 PM
    apple has a good phone with the iPhone5s....not bad, considering how small it is....powerful indeed
  • -1 Hide
    MANOFKRYPTONAK , February 22, 2014 9:33 PM
    Why don't they do a chart that includes scores changed to the same resolution? I understand why they have these original charts that show performance on the devices screen, but why don't they have a chart with that shows real hardware performance?
  • -1 Hide
    lockhrt999 , February 23, 2014 3:57 AM
    Nexus 5 is slowest example of snapdragon 800. Why didn't you use Note 3?
  • 0 Hide
    daglesj , February 23, 2014 6:26 AM
    ART or Dalvik on the KitKat Androids?
  • 0 Hide
    daglesj , February 23, 2014 6:42 AM
    Hmm well just did tests of the first two benches with my stock Nexus 4 running ART runtime and the scores were - 525 for Colossus1388 for TRex.Virtually the same as the Nexus 5 with a slower phone.Right okayyyy..
  • 0 Hide
    daglesj , February 23, 2014 6:56 AM
    The low levels were - ALU at 1800 / Alpha at 4534 / Driver at 406 / Fill at 2672.So looks like running ART can be a boost for older Android kit maybe.This was a straight install of the benchmark. No rebooting or shutting down of other apps and services.
  • -1 Hide
    Ninjawithagun , February 24, 2014 7:53 AM
    Fundamentally flawed benchmarking because the resolutions were not taken into consideration. Of course the Apple A7 processor is able to beat all the other processors because it has the least amount of graphics processing overhead. In comparison, the Nexus 7 @ 1920 x 1080 has to process 2.85 times (or 285%) more pixels per clock cycle versus the Apple A7 (1136 x 640). The benchmark scores would be nearly even (or even worse) if Apple A7 processor had to output a simliar resolution.
Display more comments