Test Setup And Benchmark Suite
Victims of the Transition to Mobile
Our start time test is now scaled back to just a single tab, started "hot". Tablet operating systems only load the page in the active tab, and additional background tabs do not load until they're selected, making the eight-tab start-up time difficult to accurately measure and basically irrelevant. Likewise, most people never actually power down their tablets, making “cold” start times irrelevant as well.
Current mobile operating systems also handle memory and multi-tasking in a very different manner than PC operating environments, with a focus on suspending background applications and single-tasking. Therefore, the memory efficiency tests from the desktop suite are also not needed.
With the loss of the 40-tab memory test and the inability to load all tabs simultaneously, the page load reliability and responsiveness tests cannot be applied to mobile platforms either. While we could include "smoothness" and/or "usability" indicators, because this is the first time we're running the WBGP on iOS, we want to see how the raw numbers stack up before injecting any subjective metrics into the equation.
Because tablets do not have on plug-ins like desktops do, the Flash, Java, and Silverlight tests from the desktop WBGP test suite are gone, too. And finally, mobile platforms do not yet support WebGL, so those tests from the regular WBGP suite were dropped as well.
Test Setup And Benchmark Suite
| Test System Specs | |
|---|---|
| Model | Apple iPad (third-generation) |
| Operating System | Apple iOS 6 |
Display9.7" LED-Backlit IPS Retina Display (2048x1536)ProcessorApple A5X ARM Cortex-A9 @ 1 GHz (dual-core)Memory1 GB LP-DDR2Storage16 GBNetworking802.11 a/b/g/nLocal Web Server SpecsOperating SystemUbuntu 12.04.1 LTS Server Edition "Precise Pangolin" (32-bit)ProcessorIntel Pentium 4 @ 2.41 GHzMotherboardBiostar P4M80-M4Memory768 MB DDR @ 333 MT/sGraphicsNvidia GeForce FX 5500 128 MB DDR (AGP)StorageWestern Digital Caviar SE WD1600AAJD, 160 GB EIDE, 7200 RPMExtra PackagesApache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH, Node.js, NPMNetwork SpecsISP ServiceCox Preferred (18 Mb/s down, 2 Mb/s up)ModemMotorola SURFboard SBS101URouterLinksys WRT54G2 V1Benchmark SuitePage Load TimeStart Time (Yahoo! homepage, Cached)Uncached Load Time (Eight Test Pages)Cached Load Time (Eight Test Pages)JavaScriptRIABench JavaScript (Nine Tests)Apple SunSpider v0.9.1 (Google Mod)Mozilla Kraken v1.1Google Octane v1FutureMark Peacekeeper v2RightWare BrowserMarkDOMMozilla Dromaeo DOM CoreCSSKaizoumarkHTML5GUIMark2 HTML5 (Four Tests)Asteroids HTML5 Canvas 2D And JavaScriptHTML5 Canvas Performance TestImpact HTML5 BenchmarkMandelbrot Set in HTML5 (FHD)Hardware AccelerationWebVizBenchStandards ConformanceHTML5Test.comThe CSS3 TestEcmascript Language test262Facebook ringmarkSecurityBrowserscope Security
The iPad, local Web server, and all software were updated as of midnight (Eastern Standard Time) on October 15th, 2012.
Now, let's break some new ground and see how the iOS Web browsers stack up against each other.
