Results: Graphics Tests
3DMark
Futuremark is a name synonymous with benchmarking GPUs, and 3DMark for mobile has quickly become a very popular tool for doing just that. 3DMark offers three main graphical benchmarks using Futuremark’s own in-house engine. The tests simulate the demands of OpenGL ES 2.0 games using shaders, particles, and physics. The first test, Ice Storm, runs at a fixed 1280x720 off-screen resolution, while the second, Ice Storm Extreme, increases the off-screen resolution to 1920x1080. Finally, Ice Storm Unlimited renders the scene at native resolution.
We added Samsung's Galaxy S4 to 3DMark because the HTC One was de-listed from Futuremark's 3DMark database for cheating; this is evident from the stark difference in Graphics scores between the two Snapdragon 600-based smartphones.
Although 3DMark was only released this May and has since received frequent updates, the Extreme test is maxed-out by the Snapdragon 800's Adreno 330 GPU. Consequently, there is no data to display for the Sony Xperia Z1. In rather stark contrast to its performance on many other GPU benchmarks, the A7's PowerVR GPU is bested by the Adreno 320, especially the Snapdragon 600-based HTC One and Samsung Galaxy S4 which are further bolstered by a faster CPU core and interconnects. Amongst the Adreno 320-equipped devices, we once again see the impact of Android 4.3 as the Nexus 4 outperforms the Find 5 and even bests the HTC One in the graphics test, despite the latter gaming this benchmark.
The results from the 3DMark Unlimited test continue to demonstrate the Nexus 4's advantage in running a stock version of Android 4.3, while the Find 5 and HTC One are disadvantaged by Android 4.1.1 and HTC Sense respectively. It would be interesting to see how these results would pan out with the Find 5 running CyanogenMod 11 (4.3) or Omnirom (4.4), utilize those performance improvements.
Basemark GUI Free
Rightware is another benchmarking stalwart, and its Rightmark toolset is well known. Basemark GUI is the company’s Android UI performance test. It’s designed to simulate the demands of intensive UI use in 3D (think applications like Google Maps). As with 3DMark, though, it’s starting to be overshadowed by more modern SoCs and implementations of Android; ever since version 4.1, more and more of Android’s UI is rendered in hardware.
This test pretty clearly indicates the age and power of the devices. Interestingly, the Find 5 fares well at its native 1920x1080, while the Nexus 4 and its lower-resolution display finish a couple of FPS back.
We switch over to the 720p off-screen BaseMark GUI test, which lifts the 60 FPS panel limit and once again demonstrates the consequence of Oppo's Find 5 running Android 4.1.1 and the HTC One running HTC Sense, since the Nexus 4 is able to outperform both.
Basemark X
Basemark X is a multi-platform benchmark based on a real game engine, Unity 4.0. It uses many of Unity’s modern features via the OpenGL ES 2.0 render path, just as a modern game would. Features like high poly count models, shaders with normal maps, complex LoD algorithms, and extensive per-pixel lighting (including directional and point light), along with a comprehensive set of post process, particle systems, and physics effects test how a modern game might run and look. It’s an aggressive test that still isn’t being maxed out by the latest mobile SoCs.
Pushing more than two megapixels punishes the Find 5 and HTC One compared to the Nexus 4's 1280x768 screen. The iPhone 5s benefits both from its powerful PowerVR GPU and the fact that its 1136x640 display adds up to less than one megapixel.
The situation changes quite a bit when you run off-screen at a common resolution. Suddenly, the A7 doesn't enjoy such a commanding lead, while the Xperia's Adreno 330 GPU flattens the other Android-based contenders. Once again 4.11 and Sense prevent either the Find 5 or HTC One from matching the cheaper, older Nexus 4.
Epic Citadel
Epic’s Unreal Engine is put to decent use in this benchmark, which simulates a reasonably simple first-person game environment. Although it is a little dated and can easily be maxed-out in Performance and High Quality modes by SoCs like the Tegra 3, this benchmark does utilize a functional game engine and is still quite useful when run in Ultra High Quality mode.
The Epic Citadel test provides us with a number of interesting observations about the devices, and about the benchmark itself. Firstly, the Nexus 4's lower screen resolution and Android 4.3 operating system allow it to outperform both the Find 5 and HTC One. With regards to the HTC One, the device scores highly in the High Performance and High Quality tests, but underperforms in the RAM-intensive Ultra High Quality mode, likely due to HTC Sense's high memory footprint.
Although Ultra High Quality mode's 8x FSAA and 16x anisotropic filtering makes Epic Citadel significantly more demanding, the test still relies on dated DirectX 8.1 effects such as baked lighting. The test’s accuracy is severely curtailed by its lack of support for fixed resolutions and off-screen rendering.
GFXBench
Kishonti GFXBench 2.7 (previously known as GLBenchmark) is a cross-platform OpenGL ES 2.0-based GPU benchmark. It not only simulates scenes that a game might render, but also runs a series of additional tests to isolate fill-rate, render accuracy, and so on. This is a demanding metric, especially the T-Rex scene, because it uses many modern effects including motion blur, parallax mapping, and complex particle systems. We will only be concentrating on four main benchmarks in the suite: T-Rex HD Off-screen, Egypt HD Off-screen, T-Rex HD On-screen, and Egypt HD On-screen.
The results from the on-screen test continue to demonstrate the excellent performance of the A7's PowerVR GPU and the substantial advantages the Nexus 4 receives on account of its lower screen resolution and Android 4.3 operating system.
In the off-screen test, however, the Nexus 4 no longer benefits from its lower screen resolution and is surpassed by the HTC One’s Snapdragon 600 SoC featuring a 1.7 GHz CPU complex (compared to the Nexus 4's 1.5 GHz SoC).
Mobile GPUMark
Silicon Studios’ Mobile GPUMark is a benchmark that employs the company’s own in-house engine called YEBIS 2. It focuses on post-processing effects like color grading, bloom, and lenticular lighting to create cinematic scenes.
The native resolution Mobile GPU Mark test provides the most significant illustration of what we are now describing as the "Nexus 4 effect", high benchmark results that can be attributed to the device's lower screen resolution and the substantial performance improvements offered by its Android 4.3 operating system. Much like Epic Citadel's Ultra High Quality mode, the memory intensive YEBIS engine has also taken a substantial toll on the HTC One's performance, which we also attribute to the RAM-heavy Sense ROM.
Wrapping up the GPU benchmarks is the off-screen YEBIS tests that provide the clearest indication of the Nexus 4 effect, since the device is able to outperform the iPhone 5s in the 400x234 and 800x450 tests and the Sony Xperia Z1 in the 800x450 test. The benchmark also makes it abundantly clear how resource-intensive Sense 5.0 actually is, since it results in the HTC One being outperformed by even the Samsung Galaxy S III in all but the 720p test.
