How We Tested
Benchmark Suite
Our current Android test line-up comprises six key sections: CPU, Web, GPU, GPGPU, Display and Battery.
| HTML5 And JavaScript Benchmarks | Browsermark 2.1, JSBench, Peacekeeper 2.0, WebXPRT 2013 |
|---|---|
| CPU Core Benchmarks | AnTuTu X (Anti-Detection), Basemark OS II Full (Anti-Detection), Geekbench 3 Pro (Anti-Detection), MobileXPRT 2013 |
| GPU Core Benchmarks | 3DMark (Anti-Detection), Basemark X 1.1 Full (Anti-Detection), GFXBench 3.0 Corporate |
| GPGPU Benchmarks | CompuBenchRS |
| Display Measurements | Brightness(Min/Max), Black Level, Contrast Ratio, Gamma, Color Temperature, Color Gamut (sRGB/AdobeRGB) |
| Battery Tests | Basemark OS II Full (Anti-Detection), BatteryXPRT 2014, GFXBench 3.0 Corporate |
Test Methodology
All handsets are benchmarked on a fully updated copy of the device's stock software. The table below lists other common device settings that we standardize to before testing.
| Bluetooth | Off |
|---|---|
| Brightness | 200 nits |
| Cellular | SIM card removed |
| Display Mode | Device Default (non-adaptive) |
| Location Services | Off |
| Power | Battery |
| Sleep | Never (or longest available interval) |
| Volume | Muted |
| Wi-Fi | On |
Furthermore, for browser-based testing on Android, we're employing a static version of the Chromium-based Opera in order to keep the browser version even across all devices. Due to platform restrictions, Safari is the best choice for iOS-based devices, while Internet Explorer is the only game in town on Windows RT.
Comparison System Specs
Qualcomm had a strong year in 2014, with nearly every flagship smartphone running a Snapdragon SoC. The Snapdragon 801 was particularly popular, showing a good balance between performance and efficiency. Its ubiquity isn't necessarily bad for consumers, but it does make our benchmark charts rather boring.
The table below contains all the pertinent technical specifications for today’s comparison units:
Other than the iPhone 6, all of the Android phones we'll be comparing use the Snapdragon 801 SoC. The Galaxy S5 uses the highest-clocked 8974AC version just like the G3, One and Z3. Only the HTC One (M8) uses the lower-clocked 8974AB variant, which should give it slightly lower CPU performance.
Again, other than the iPhone 6 with its PowerVR GPU, all of the Android phones use the Adreno 330 graphics engine. The only differentiator for GPU performance should be the G3's higher WQHD screen resolution.
We should see some variation in longevity, though. Both the G3 and OnePlus One have slightly larger batteries than the S5, but they also have larger screens to power. The iPhone 6 has by far the smallest battery. Will its smaller, lower-resolution screen make up for this deficit? The Sony Z3 has a bigger battery than the S5 and almost the same size screen. This should give the Z3 an edge in battery life.
