Test Results: GPU Core Benchmarks
Futuremark has become a name synonymous with benchmarking, and the company's latest iteration of 3DMark offers three main graphical benchmarks: Ice Storm, Cloud Gate and Fire Strike. Currently, the DirectX 9-level Ice Storm tests are cross-platform for Windows, Windows RT, Android and iOS.
Ice Storm simulates the demands of OpenGL ES 2.0 games using shaders, particles and physics via the company's in-house engine. Although it was just released in May of 2013, the on-screen portions of Ice Storm have already been outpaced by modern mobile chipsets, with Nvidia's Tegra 4 and Qualcomm's Snapdragon 800 both easily maxing-out the Extreme version (1080p with high-quality textures). However, Ice Storm Unlimited, which renders the scene off-screen at 720p, is still a good gauge of GPU-to-GPU performance.
The upgraded PowerVR GX6450 in the A8 SoC gives the iPhone 6 Plus a 24% boost over the iPhone 5s in the Graphics test. Imagination's latest GPU performs 19% better than the Adreno 420 in the Galaxy Note 4. Since this benchmark is heavily influenced by shader performance, it implies the GX6450 has the more robust shading architecture. We’ll see if this holds true for the remaining benchmarks.
The 6 Plus shows a 5% advantage over the iPhone 6 in the Graphics test, similar to the 7% gap in Basemark OS II Graphics.
The new iPhone's Physics score is less impressive. While it nets a 19% improvement over the 5s, the A8 still trails significantly behind Snapdragon.
Since Cyclone does so well in other CPU benchmarks, it's curious that it underperforms in this physics test. After the A7 SoC appeared last year, Futuremark, the developer of 3DMark, performed a thorough investigation of this issue and discovered that Cyclone's performance suffered when "dealing with non-sequential data structures with memory dependencies." Basically, the A7's memory controller is optimized for sequential rather than random access. Also, working with data sets where a variable depends on one or more other variables within the same set makes executing instructions out-of-order more difficult, nullifying its IPC advantage.
Basemark X 1.1
Based on the Unity 4.0 game engine, Rightware’s Basemark X is a cross-platform graphics benchmark for Android, iOS and Windows Phone 8. This test utilizes Unity’s modern features via the OpenGL ES 2.0 render path to simulate how a modern game might look and run. Basemark X is an aggressive metric that still hasn’t been maxed out by the latest mobile SoCs.
Before discussing the Basemark X results, please note that the on+screen results for the iPhone 6 and 6 Plus are not accurate, since the current version of this benchmark doesn’t support their native resolutions. Instead, both new iPhones render at the native resolution of the 5s (640x1136). As explained to us by Rightware, keeping the v1.1.1 scores comparable to v1.1 means using Xcode 5 for the iOS version, which doesn’t support the new iPhone 6 resolutions. However, the Overall scores are not affected by this issue, since they are based on the off-screen rendering tests.
The iPhone 6 sees a 17% performance increase over the 5s and places ahead of the Snapdragon based phones in the Basemark X Overall score. Looking at the off-screen results shows both of the new iPhones and the 5s scoring higher in Hangar. The scores for Dunes are grouped more tightly, with the Adreno 420 in the Galaxy Note 4 showing a slight advantage.
There are two interesting anomalies in these results. First, the Galaxy Note 4 scores lower than both the Note 3 and HTC One (M8) in the Hangar test. In our first look at the Adreno 420 running in a Qualcomm development tablet, it consistently outperformed the Adreno 330 in off-screen tests. We’ll look more closely at the Note 4 in a future review.
Once again the 6 Plus is a bit faster than the iPhone 6, but only by 3% in off-screen rendering. They perform essentially the same in the on-screen test.
Things look a bit different when running the benchmark at the high quality settings. In the off+screen test, the iPhone 6 scores 15% higher in Dunes and 26% higher in Hangar than the 5s. The Adreno 420 in the Note 4 performs much better, topping the latest PowerVR GPU in Dunes, but falling short in Hangar. It does outperform the Adreno 330 devices in this test.
The 6 Plus is 6% faster in both Dunes and Hangar off-screen and 3-5% faster in the on-screen tests. A definite pattern is emerging.
GFXBench 3.0 Corporate
Kishonti GFXBench 3.0 is a cross-platform GPU benchmark supporting both the OpenGL ES 2.0 and OpenGL ES 3.0 APIs. It comprises both “high-level” game-like scenarios, along with more “low-level” tests designed to measure specific subsystems.
Among the high-level tests are Manhattan and T-Rex. Manhattan is a modern, complex OpenGL ES 3.0-based test, while the OpenGL ES 2.0-level T-Rex is a holdover from GFXBench v2.7.
The low-level tests include Fill, which measures fill rate by rendering four layers of compressed textures; Alpha Blending, a test that renders layers of semi-transparent quads using high-resolution, uncompressed textures; ALU, for measuring shader compute performance; and Driver Overhead, which measures the CPU overhead of the graphics driver and API by making a lot of draw calls and state changes.
See GFXBench 3.0: A Fresh Look At Mobile Benchmarking for a complete test-by-test breakdown of this benchmark.
The iPhone 6 is 40% faster than the 5s in the demanding Manhattan test, its largest delta yet. The PowerVR GX6450 and Adreno 420 are equals in this benchmark.
The 6 Plus maintains a 5.5% lead over the iPhone 6. This definitely doesn’t look like a mere coincidence.
This is our first look at on-screen rendering performance and finally we see the effects of the larger iPhone 6 and 6 Plus displays. The iPhone 6 is about on par with the smaller 5s, its more powerful GPU compensating for the increase in pixels. The 6 Plus however, sees a regression in performance relative to the 5s, its PowerVR GX6450 insufficient to compensate for the HD resolution. The Note 4 suffers a similar fate as Qualcomm's Adreno 420 struggles with the QHD display.
Another impressive gain for the iPhone 6, beating the 5s by 65%, but trailing the 6 Plus by a now-familiar 5% margin.
The Note 4 doesn’t do well, experiencing a great deal of thermal throttling.
It looks like the PowerVR GX6450 is well-optimized for the OpenGL ES 2.0-based T-Rex workload. Despite dealing with more pixels, both the iPhone 6 and 6 Plus manage to outperform the 5s.
In the Alpha Blending test, which deals with reading high-resolution, uncompressed textures from RAM, Snapdragon 805’s 25.6GB/s of memory bandwidth—compared to Snapdragon 801 (14.9GB/s), Snapdragon 800 (12.8GB/s) and iPhone 5s/6 (12.8GB/s)—gives it the advantage. All of the devices appear to be constrained by memory bandwidth.
Snapdragon shows a clear affinity for the shader-intensive ALU test, turning the tables on the iPhone. The iPhone 6 shows a 14% gain relative to the 5s, and the 6 Plus is 10% faster still.
The finishing order in the Fill rate test, which also depends on memory bandwidth, is similar to Alpha Blending. The PowerVR G6430 in the 5s is hampered by weak fill rate performance. While this metric improves by 34% in the iPhone 6, it still trails all of the Snapdragon-powered devices. Again, we see the 6 Plus performing 10% better than the iPhone 6.
The new PowerVR GX6450 in the iPhone 6 is a definite upgrade over the previous-generation PowerVR G6430. Performance improvements range from 15-65% with an average around 30%. With nearly 3.8x as many pixels to process for the 6 Plus (the 6 Plus renders at 2208x1242 and then scales down to 1920x1080), the increase in performance doesn’t match the increase in pixels. This results in some performance regression compared to the iPhone 5s.
In all of the graphics tests, we consistently see the 6 Plus with a 5-10% performance advantage over the iPhone 6. If this occurred in one or two benchmarks, it could easily be written off as an anomaly. But there appears to be a legitimate difference here. The A8 SoCs in both the iPhone 6 and 6 Plus share the same part number, so there’s no hardware differences between them. There are at least two different memory suppliers, but our iPhone 6 actually has a bit more bandwidth than our 6 Plus, so no advantage there. It seems possible, then, that the GPU in the 6 Plus runs at a higher frequency. The 6 Plus is responsible for far more pixels than the iPhone 6, so it can definitely benefit from a frequency boost, and with a larger battery and more aluminum surface area, it can handle the extra power and thermal requirements. Since the performance gain is relatively small, the increase could only be about 50MHz or 11%, assuming a base clock of 450MHz. This would certainly explain the performance delta, but is an extra 50MHz worth the cost in battery life when there are 2.7x more pixels to render?
Turning our attention to Power VR’s nearest competitor, the GX6450 shares a similar relationship with the Adreno 420 as the G6430 did with the Adreno 330, each gaining an advantage in certain benchmarks. Snapdragon 805’s extra memory bandwidth gives the Adreno 420 a big advantage when pulling textures from memory, however the GX6450 seems to have a slight edge in shader performance.