Web Browser Grand Prix VIII: Chrome 16, Firefox 9, And Mac OS X

WBGP VIII Test Setup

Hardware Setup

Software Setup

Our test installations are freshly installed and fully updated as of midnight on December 20^th. Power management and automatic updating is disabled before testing. The Web browsers and additional software, along with the exact version numbers tested, are listed in the table below.

Network Setup

The following table contains the system specs of the local Web server used for our startup time tests, the page load time tests, and JSGameBench.

The table below holds additional information on the test network.

 
While we typically use a hard-wired Ethernet connection, due to the MacBook Air being the test system, we're debuting the use of Wi-Fi in the Web Browser Grand Prix.

Test Setup

We restart the computer and allow it to idle for a few minutes before benchmarking the next browser. Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variance.

All tests are placed into one of four groups: core, observation, dated, and quarantine. Core tests are considered current. These tests are usually trusted industry standards or our own creations, and they make up the core of the WBGP suite. Tests that are either generally unknown, mostly untested, or just too bleeding-edge are placed under observation. Tests classified as dated are either outdated, losing relevance, or otherwise need replacing. We are actively seeking community feedback and contributions regarding alternatives to these benchmarks. The final group is for quarantined benchmarks. Benchmarks find their way into quarantine by delivering dubious results or by being gamed. Whenever benchmarks that test the same thing yield conflicting results, more weight is given to tests with a better rating when creating the analysis tables.

The table below lists all 53 of the tests currently in our suite (along with a version number and link, where applicable), number of iterations performed, and current rating:

Legend

Throughout this article there are a ton of charts, many of which house data from both Windows 7 and Mac OS X. The data in these charts is sorted by the Windows 7 score, with the best performer on top and the worst at the bottom. Windows 7 scores are represented by blue bars and OS X scores are in green. If an OS X browser outperforms all of the Windows 7 browsers, we'll indicate this occurrence by changing the color of the winning OS X browser from green to red.

Now, let's fire off the starting pistol and let the race begin!