Benchmark Suite, Methodology, And Test System Specs
Benchmark Suite
Comparing smartphone performance is a little more challenging than comparing PC components, since we have to find benchmarks that work across Windows Phone 8, Android, and iOS. Of the three, Windows Phone 8 has the lowest number of cross-platform testing apps, which makes sense when you consider its marketshare. Not only that, but particular phone models are sometimes incompatible with specific metrics, throwing yet another wrench in the pipe.
| CPU Core & System Tests | Rightware Basemark OS II Full |
|---|---|
| GPU Core & Graphics Tests | Rightware Basemark X 1.1 Full |
| Web Tests | Rightware Browsermark 2.0 |
| Futuremark Peacekeeper 2.0 | |
| Principled Technologies WebXPRT 2013 | |
| Display Measurements | Minimum & Maximum Brightness |
| Black Level | |
| Contrast Ratio | |
| Gamma | |
| Color Temperature | |
| sRGB & AdobeRGB Color Gamut Volume | |
| Battery Tests | Rightware Basemark OS II Full |
| Camera Tests | Rightware Basemark OS II Free |
Further complicating today's performance analysis, we test all of our smartphones at a calibrated 200 nits of screen brightness to ensure comparable battery life and display measurements. Unfortunately, Windows Phone 8 does not have an adjustable screen brightness slider; it is limited to three factory preset settings: low, medium, and high. To provide comparable battery life and display measurements, we had to test at those three brightness levels and use them to extrapolate a 200 nit calibrated result. It's not ideal, but it's the best possible method we have at our disposal.
Test System Specs
Despite these challenges, Tom's Hardware is all about objective measurements and we've done our absolute best to provide the most cross-platform tests for as many platforms and SoCs as possible.
The iPhone 5s represents ARM v8, Meizu's MX3 represents the Exynos 5 Octa, Xiaomi's Mi3 represents the Nvidia Tegra 4, Google's Nexus 5 represents Snapdragon 800 performance on Android, and the HTC 8x represents the previous generation of Windows Phone 8 device performance with a dual-core Snapdragon S4 Plus at 1.5 GHz.
| Device | SoC | CPU | GPU | RAM | Display | Operating System |
|---|---|---|---|---|---|---|
| Apple iPhone 5s | Apple A7 | Apple Cyclone (dual-core) @ 1.3 GHz | Imagination Technologies PowerVR G6430 (four-cluster) @ 200 MHz | 1 GB DDR3 | 4" IPS @ 1136x640 (326 PPI) | Apple iOS 7.1 |
| Google Nexus 5 | Qualcomm Snapdragon 800 (MSM8974AA) | Qualcomm Krait 400 (quad-core) @ 2.26 GHz | Qualcomm Adreno 330 (quad-core) @ 450 MHz | 2 GB DDR3 @ 800 MHz | 4.95" IPS @ 1920x1080 (445 PPI) | Google Android 4.4.4 (GPe w/GEL) |
| Meizu MX3 | Samsung Exynos 5 Octa (5410) | ARM Cortex-A15 (quad-core) @ 1.6 GHzARM Cortex-A7 (quad-core) @ 1.2 GHz | Imagination Technologies PowerVR SGX544MP3 (triple-core) @ 532 MHz | 2 GB DDR3 | 5" IPS @ 1920x1080 (412 PPI) | Google Android 4.2.1 (FlymeOS 3.1.1) |
| Xiaomi Mi3 | Nvidia Tegra 4 (T114) | ARM Cortex-A15 (quad-core) @ 1.8 GHz | Nvidia GeForce ULP (72-core) @ 672 MHz | 2 GB DDR3 | 5" IPS @ 1920x1080 (441 PPI) | Google Android 4.2.1 (MIUI 4.1.24) |
| HTC 8x | Qualcomm Snapdragon S4 Plus | Qualcomm Krait (dual-core) @ 1.5 GHz | Qualcomm Adreno 225 | 2 GB DDR2 | 4.5" AMOLED @1280x768 (342 PPI) | Microsoft Windows Phone 8 |
| Nokia Lumia Icon/930 | Qualcomm Snapdragon 800(MSM8974VV) | Qualcomm Krait 400 (quad-core) @ 2.15 GHz | Qualcomm Adreno 330 @ 450 MHz | 2 GB DDR3 @ 800 MHz | 5" AMOLED @ 1920x1080 (441 PPI) | Microsoft Windows Phone 8 |
