Sign in with
Sign up | Sign in

Test Setup And Benchmark Suite

Which Browser Should You Be Running On Your iPad And iPhone?
By

Victims of the Transition to Mobile

Our start time test is now scaled back to just a single tab, started "hot". Tablet operating systems only load the page in the active tab, and additional background tabs do not load until they're selected, making the eight-tab start-up time difficult to accurately measure and basically irrelevant. Likewise, most people never actually power down their tablets, making “cold” start times irrelevant as well.

Current mobile operating systems also handle memory and multi-tasking in a very different manner than PC operating environments, with a focus on suspending background applications and single-tasking. Therefore, the memory efficiency tests from the desktop suite are also not needed.

With the loss of the 40-tab memory test and the inability to load all tabs simultaneously, the page load reliability and responsiveness tests cannot be applied to mobile platforms either. While we could include "smoothness" and/or "usability" indicators, because this is the first time we're running the WBGP on iOS, we want to see how the raw numbers stack up before injecting any subjective metrics into the equation.

Because tablets do not have on plug-ins like desktops do, the Flash, Java, and Silverlight tests from the desktop WBGP test suite are gone, too. And finally, mobile platforms do not yet support WebGL, so those tests from the regular WBGP suite were dropped as well.

Test Setup And Benchmark Suite

Test System Specs
Model
Apple iPad (third-generation)
Operating System
Apple iOS 6
Home ScreenHome Screen
Display
9.7" LED-Backlit IPS Retina Display (2048x1536)
Processor
Apple A5X ARM Cortex-A9 @ 1 GHz (dual-core)
Memory
1 GB LP-DDR2
Storage
16 GB
Networking
802.11 a/b/g/n
Local Web Server Specs
Operating System
Ubuntu 12.04.1 LTS Server Edition "Precise Pangolin" (32-bit)
Processor
Intel Pentium 4 @ 2.41 GHz
Motherboard
Biostar P4M80-M4
Memory
768 MB DDR @ 333 MT/s
Graphics
Nvidia GeForce FX 5500 128 MB DDR (AGP)
Storage
Western Digital Caviar SE WD1600AAJD, 160 GB EIDE, 7200 RPM
Extra Packages
Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH, Node.js, NPM
Network Specs
ISP Service
Cox Preferred (18 Mb/s down, 2 Mb/s up)
Modem
Motorola SURFboard SBS101U
Router
Linksys WRT54G2 V1
Benchmark Suite
Page Load Time
Start Time (Yahoo! homepage, Cached)
Uncached Load Time (Eight Test Pages)
Cached Load Time (Eight Test Pages)
JavaScript
RIABench JavaScript (Nine Tests)
Apple SunSpider v0.9.1 (Google Mod)
Mozilla Kraken v1.1
Google Octane v1
FutureMark Peacekeeper v2
RightWare BrowserMark
DOM
Mozilla Dromaeo DOM Core
CSS
Kaizoumark
HTML5
GUIMark2 HTML5 (Four Tests)
Asteroids HTML5 Canvas 2D And JavaScript
HTML5 Canvas Performance Test
Impact HTML5 Benchmark
Mandelbrot Set in HTML5 (FHD)
Hardware Acceleration
WebVizBench
Standards ConformanceHTML5Test.com
The CSS3 Test
Ecmascript Language test262
Facebook ringmark
SecurityBrowserscope Security


The iPad, local Web server, and all software were updated as of midnight (Eastern Standard Time) on October 15th, 2012.

Now, let's break some new ground and see how the iOS Web browsers stack up against each other.

React To This Article