Sign in with
Sign up | Sign in

WBGP VIII Test Setup

Web Browser Grand Prix VIII: Chrome 16, Firefox 9, And Mac OS X
By

Hardware Setup

Test System Specs
Operating System 1
Microsoft Windows 7 Ultimate (64-bit)
Operating System 2
Apple Mac OS X Lion (64-bit)
Model
Apple MacBook Air 11.6-inch (late 2011)
Processor
Intel Core i7 @ 1.8 GHz (dual-core)
Memory
4 GB DDR3 @ 1333 MT/s
Graphics
Intel HD Graphics 3000 384 MB DDR3 SDRAM
Storage
128 GB SSD


Behold The Glory!Behold The Glory!

Software Setup

Our test installations are freshly installed and fully updated as of midnight on December 20th. Power management and automatic updating is disabled before testing. The Web browsers and additional software, along with the exact version numbers tested, are listed in the table below.

Software
Version
Chrome16.0.912.63 m
Firefox
9.0.1
Internet Explorer
9.0.8112.16421
Opera
11.60 (build 1185)
Safari
5.1.2 (7534.5.2.7)
Adobe Flash11.1.102.55 (64-bit)
Microsoft Silverlight5.0.61118.0
Oracle Java6.0.30


The Windows 7 Test InstallationThe Windows 7 Test Installation

The Mac OS X Lion Test InstallationThe Mac OS X Lion Test Installation

Network Setup

The following table contains the system specs of the local Web server used for our startup time tests, the page load time tests, and JSGameBench.

Local Web Server Specs
Operating System
Ubuntu 10.04 LTS Server Edition "Lucid Lynx" (32-bit)
Processor
AMD Athlon @ 1150 MHz
Motherboard
Soyo Dragon Platinum
Memory
512 MB DDR
Graphics
AMD Radeon 9550, 256 MB GDDR
Storage
40 GB Western Digital HDD WD400BB
Optical
Samsung DVD-ROM SD-616T
Extra Packages
Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH


The table below holds additional information on the test network.

Network Specs
ISP Service
Cox Premium (28 Mb/s down, 5 Mb/s up)
Modem
Motorola SURFboard SBS101U
Router
Linksys WRT54G2 V1

 
While we typically use a hard-wired Ethernet connection, due to the MacBook Air being the test system, we're debuting the use of Wi-Fi in the Web Browser Grand Prix.

Test Setup

We restart the computer and allow it to idle for a few minutes before benchmarking the next browser. Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variance.

All tests are placed into one of four groups: core, observation, dated, and quarantine. Core tests are considered current. These tests are usually trusted industry standards or our own creations, and they make up the core of the WBGP suite. Tests that are either generally unknown, mostly untested, or just too bleeding-edge are placed under observation. Tests classified as dated are either outdated, losing relevance, or otherwise need replacing. We are actively seeking community feedback and contributions regarding alternatives to these benchmarks. The final group is for quarantined benchmarks. Benchmarks find their way into quarantine by delivering dubious results or by being gamed. Whenever benchmarks that test the same thing yield conflicting results, more weight is given to tests with a better rating when creating the analysis tables.

The table below lists all 53 of the tests currently in our suite (along with a version number and link, where applicable), number of iterations performed, and current rating:

Tom's Hardware Web Browser Grand Prix Test Suite v8.0
Test Name
Iterations
Rating
Performance Tests (44)
Cold Startup Time: Single Tab
3
Core
Cold Startup Time: Eight Tabs3
Core
Hot Startup Time: Single Tab3
Core
Hot Startup Time: Eight Tabs
3
Core
Uncached Page Load Times (9 Test Pages)
5
Core
Cached Page Load Times (9 Test Pages)
5
Core
Kraken v1.1
2
Core
Google Kraken v1.1 Mod2
Observation
SunSpider v0.9.12
Observation
Google SunSpider v0.91 Mod2
Observation
FutureMark Peacekeeper 2.02
Core
Dromaeo DOM2
Core
Maze Solver5
Core
GUIMark2 Flash Vector Charting3
Core
GUIMark2 Flash Bitmap Gaming3
Core
GUIMark2 Flash Text Columns3
Core
Flash Benchmark 2008 v1.09.1
2
Core
GUIMark Java3
Dated
Encog Silverlight3
Dated
Facebook JSGameBench v4.12
Core
GUIMark 2 HTML5 Vector Charting (1 pixel variant)
3
Core
GUIMark 2 HTML5 Bitmap Gaming
3
Core
GUIMark 2 HTML5 Text Columns
3
Core
Asteroids HTML5 Canvas 2D And JavaScript2
Core
Psychedelic Browsing
2
Observation
Hardware Acceleration Stress Test
2
Dated
Mozilla WebGL FishIE
5
Observation
WebGL Solar System5
Observation
Efficiency Benchmarks (5)
Memory Usage: Single Tab
3
Core
Memory Usage: 40 Tabs
3
Core
Memory Management: -39 Tabs
3
Core
Memory Management: -39 Tabs (extra 2 minutes)
3
Core
Reliability Benchmarks (1)
Proper Page Loads
3
Core
Conformance Benchmarks (3)
Ecma test2621
Core
Peacekeeper 2.0 HTML5 Capabilities1
Core
HTML5Test.com1
Core


Legend

Throughout this article there are a ton of charts, many of which house data from both Windows 7 and Mac OS X. The data in these charts is sorted by the Windows 7 score, with the best performer on top and the worst at the bottom. Windows 7 scores are represented by blue bars and OS X scores are in green. If an OS X browser outperforms all of the Windows 7 browsers, we'll indicate this occurrence by changing the color of the winning OS X browser from green to red.

Now, let's fire off the starting pistol and let the race begin!

React To This Article