Sign in with
Sign up | Sign in
Web Browser Grand Prix VIII: Chrome 16, Firefox 9, And Mac OS X
By ,
1. Web Browser Grand Prix VIII

Last August, we ran the Web Browser Grand Prix (WBGP) in an entirely new venue. The title was Web Browser Grand Prix VI: Firefox 6, Chrome 13, Mac OS X Lion, and we used the same desktop test system we always use, but Hackintoshed. Despite the fact that an OS X browser took first place in nearly one-third of the tests, some of the more, er, enthusiastic Mac fans weren't very happy with our platform choice. They set down their Starbucks double venti caramel macchiatos to complain, “But Adam, that's not a real Macintosh, so of course OS X on a PC is going to suck compared to Windows” and “why didn't you run it on a MacBook with Parallels?” and (my all-time favorite) "it's not fair!"

The short answer is that we don't have a ton of Macs in our PC-centric labs. Nevertheless, we went out and bought a brand-new 11” MacBook Air specifically to satisfy the folks who want to see browsers tested on their native platforms. Welcome to Web Browser Grand Prix VIII, the Mac rematch!

But before we dig our heels into this Mac versus PC Web browser battle royale, let's get everyone caught up on the latest events and ensuing drama. More than two months have passed since we published Web Browser Grand Prix 7: Firefox 7, Chrome 14, Opera 11.51, and a lot has happened in the browser wars since then.

Recent Events

October 9th 2011: On this day, Chrome briefly became the world's second most popular Web browser (according to StatCounter).

October 11th 2011: Opera floats the idea of replacing the scrollbar with set length pages which can be turned, like e-books.

October 15th 2011: Google announces that Chrome has reached 200 million active users.

October 26th 2011: Google releases Chrome 15.

October 28th 2011: Mozilla and Microsoft team up to launch a special version of Firefox with Bing as the default search provider.

November 1st 2011: Opera releases an impressive WebGL demo code-named Odin, requiring a WebGL-capable browser.

November 3rd 2011: Chrome finishes October just shy of 25% overall Web browser market share (according to StatCounter).

November 7th 2011: Mozilla releases Firefox 8.

November 10th 2011: Mozilla celebrates the 7th anniversary of Firefox.

December 2nd 2011: Google Chrome overtakes Mozilla Firefox in market share to become the second most popular Web browser, behind Internet Explorer

December 6th 2011: Opera releases version 11.60

December 14th 2011: Google updates Chrome to version 16.

December 16th 2011: Windows Update now auto-updates Internet Explorer to the latest version. Meanwhile, Chrome 15 edges out IE8 as the most widely used Web browser version, although Microsoft claims IE9 holds that title (when only looking at Windows 7 installations).

December 16th 2011: Mozilla releases a developer preview of Apps project and hints at Firefox as a gaming platform.

December 20th 2011: Mozilla releases Firefox 9.

December 21st 2011: Mozilla renews its search bar deal with Google for another three years, at three times the cost of the previous arrangement. There has been speculation that the increase is due to a bidding war with Microsoft Bing.

January 4th 2012: Use of IE6 drops below 1% in the US, and Microsoft is thrilled. "IE6 has been the punch line of browser jokes for a while, and we’ve been as eager as anyone to see it go away," writes Roger Capriotti, director of Internet Explorer marketing.

Recent Drama

September 26th 2011: Google launches a prime time television commercial for Chrome featuring Angry Birds.

Google Chrome: Angry Birds

September 30th 2011: Microsoft Security Essentials mistakenly classifies Chrome and Firefox as malware.

October 11th 2011: Microsoft concocts a browser security test in which IE9 is “proven” to be more secure than both Mozilla Firefox and Google Chrome. Apple Safari and Opera were left out of the testing.

November 4th 2011: Alex Faaborg, the lead UI designer of Firefox announces that he is leaving Mozilla. He is the third high-profile employee to do so in 2011. Another Mozilla developer admits Firefox needs to support importing of Chrome bookmarks. Also calls Add-ons for the Windows version of Firefox “awful.”

November 15th 2011: Mozilla adopts three endangered Red Panda (Firefox) cubs and yet again sets up a live 24-hour feed of their pen on Firefox Live. This time the site contains a warning for users of competing Web browsers.

December 12th 2011: Google funds a Web browser security study in which Chrome is found to be the safest browser. Apple Safari and Opera were left out of the testing.

December 27th 2011: In response to the renewal of the Firefox/Google search bar deal, Chrome developer Peter Kastings says in an interview that “the primary goal of Chrome is to make the web advance as much and as quickly as possible.” Firefox product manager Asa Dotzler counters that “This is Google's business, they sell ads...”

Wow, what a couple of months! Mozilla seems to realize that Chrome is the real threat, as Google becomes the number two Web browser, and all of the top three continue with their antics. Meanwhile Apple remains silent on the browser front, and Opera has some really big changes in store for 2012.

2. Chrome, Firefox, Internet Explorer, Opera, And Safari
Web Browser Grand Prix Champions

The table below lists all of the previous Web Browser Grand Prix winners, with links to their respective articles.

Web Browser Grand Prix
Champion
Web Browser Grand Prix: The Top Five, Tested And Ranked
Web Browser Grand Prix 2: The Top Five Tested And Ranked
Web Browser Grand Prix 2: Running The Linux Circuit
Web Browser Grand Prix 3:  IE9 Enters The Race
Web Browser Grand Prix 4: Firefox 4 Goes Final
Web Browser Grand Prix 5: Opera 11.50, Firefox 5, And Chrome 12
Web Browser Grand Prix VI: Firefox 6, Chrome 13, Mac OS X Lion
Web Browser Grand Prix 7: Firefox 7, Chrome 14, Opera 11.51

Web Browser Grand Prix 7 Contenders





Chrome
Vendor:
Google
Debut:
9/02/2008
Current Version:
16
Layout Engine:
WebKit
JavaScript Engine:
V8
Supported Platforms:
Windows, Mac, Linux
HTML5 Hardware Acceleration:
No
WebGL:
Yes
WBGP Wins:
4
Download Chrome!

Chrome 16 on Windows 7Chrome 16 on Windows 7Chrome 16 on Mac OS X LionChrome 16 on Mac OS X Lion




Firefox
Vendor:
Mozilla
Debut:
11/09/2004
Current Version:
9
Layout Engine:
Gecko 2.0
JavaScript Engine:
JaegerMonkey
Supported Platforms:
Windows, Mac, Linux
HTML Hardware Acceleration:
Yes
WebGL:
Yes
WBGP Wins:
1
Download Firefox!

Firefox 9 on Windows 7Firefox 9 on Windows 7Firefox 9 on Mac OS X LionFirefox 9 on Mac OS X Lion




Internet Explorer
Vendor:
Microsoft
Debut:
8/16/1995
Current Version:
9
Layout Engine:
Trident
JavaScript Engine:
Chakra
Supported Platforms:
Windows (Vista and 7)
HTML5 Hardware Acceleration:
Yes
WebGL:
No
WBGP Wins:
2
Download Internet Explorer!

Internet Explorer 9Internet Explorer 9




Opera
Vendor:
Opera Software
Debut:
12/09/1996
Current Version:
11.60
Layout Engine:
Presto
JavaScript Engine:
Carakan
Supported Platforms:
Windows, Mac, Linux
HTML5 Hardware Acceleration:
No
WebGL:
No
WBGP Wins:
1
Download Opera!

Opera 11.60 on Windows 7Opera 11.60 on Windows 7Opera 11.60 on Mac OS X LionOpera 11.60 on Mac OS X Lion




Safari
Vendor:
Apple
Debut:
1/07/2003
Current Version:
5.1.2
Layout Engine:
WebKit 2
JavaScript Engine:
Nitro
Supported Platforms:
Windows, Mac
HTML5 Hardware Acceleration:
Mac-only
WebGL:
No
WBGP Wins:
1
Download Safari!

Safari 5.1.2 on Windows 7Safari 5.1.2 on Windows 7Safari 5.1.2 on Mac OS X LionSafari 5.1.2 on Mac OS X Lion

This article represents the biggest departure from our traditional WBGP equipment since we introduced the Core i5 system in Web Browser Grand Prix 3: IE9 Enters The Race, so let's take a look at the platforms in question.

3. WBGP VIII Test Setup

Hardware Setup

Test System Specs
Operating System 1
Microsoft Windows 7 Ultimate (64-bit)
Operating System 2
Apple Mac OS X Lion (64-bit)
Model
Apple MacBook Air 11.6-inch (late 2011)
Processor
Intel Core i7 @ 1.8 GHz (dual-core)
Memory
4 GB DDR3 @ 1333 MT/s
Graphics
Intel HD Graphics 3000 384 MB DDR3 SDRAM
Storage
128 GB SSD


Behold The Glory!Behold The Glory!

Software Setup

Our test installations are freshly installed and fully updated as of midnight on December 20th. Power management and automatic updating is disabled before testing. The Web browsers and additional software, along with the exact version numbers tested, are listed in the table below.

Software
Version
Chrome16.0.912.63 m
Firefox
9.0.1
Internet Explorer
9.0.8112.16421
Opera
11.60 (build 1185)
Safari
5.1.2 (7534.5.2.7)
Adobe Flash11.1.102.55 (64-bit)
Microsoft Silverlight5.0.61118.0
Oracle Java6.0.30


The Windows 7 Test InstallationThe Windows 7 Test Installation

The Mac OS X Lion Test InstallationThe Mac OS X Lion Test Installation

Network Setup

The following table contains the system specs of the local Web server used for our startup time tests, the page load time tests, and JSGameBench.

Local Web Server Specs
Operating System
Ubuntu 10.04 LTS Server Edition "Lucid Lynx" (32-bit)
Processor
AMD Athlon @ 1150 MHz
Motherboard
Soyo Dragon Platinum
Memory
512 MB DDR
Graphics
AMD Radeon 9550, 256 MB GDDR
Storage
40 GB Western Digital HDD WD400BB
Optical
Samsung DVD-ROM SD-616T
Extra Packages
Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH


The table below holds additional information on the test network.

Network Specs
ISP Service
Cox Premium (28 Mb/s down, 5 Mb/s up)
Modem
Motorola SURFboard SBS101U
Router
Linksys WRT54G2 V1

 
While we typically use a hard-wired Ethernet connection, due to the MacBook Air being the test system, we're debuting the use of Wi-Fi in the Web Browser Grand Prix.

Test Setup

We restart the computer and allow it to idle for a few minutes before benchmarking the next browser. Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variance.

All tests are placed into one of four groups: core, observation, dated, and quarantine. Core tests are considered current. These tests are usually trusted industry standards or our own creations, and they make up the core of the WBGP suite. Tests that are either generally unknown, mostly untested, or just too bleeding-edge are placed under observation. Tests classified as dated are either outdated, losing relevance, or otherwise need replacing. We are actively seeking community feedback and contributions regarding alternatives to these benchmarks. The final group is for quarantined benchmarks. Benchmarks find their way into quarantine by delivering dubious results or by being gamed. Whenever benchmarks that test the same thing yield conflicting results, more weight is given to tests with a better rating when creating the analysis tables.

The table below lists all 53 of the tests currently in our suite (along with a version number and link, where applicable), number of iterations performed, and current rating:

Tom's Hardware Web Browser Grand Prix Test Suite v8.0
Test Name
Iterations
Rating
Performance Tests (44)
Cold Startup Time: Single Tab
3
Core
Cold Startup Time: Eight Tabs3
Core
Hot Startup Time: Single Tab3
Core
Hot Startup Time: Eight Tabs
3
Core
Uncached Page Load Times (9 Test Pages)
5
Core
Cached Page Load Times (9 Test Pages)
5
Core
Kraken v1.1
2
Core
Google Kraken v1.1 Mod2
Observation
SunSpider v0.9.12
Observation
Google SunSpider v0.91 Mod2
Observation
FutureMark Peacekeeper 2.02
Core
Dromaeo DOM2
Core
Maze Solver5
Core
GUIMark2 Flash Vector Charting3
Core
GUIMark2 Flash Bitmap Gaming3
Core
GUIMark2 Flash Text Columns3
Core
Flash Benchmark 2008 v1.09.1
2
Core
GUIMark Java3
Dated
Encog Silverlight3
Dated
Facebook JSGameBench v4.12
Core
GUIMark 2 HTML5 Vector Charting (1 pixel variant)
3
Core
GUIMark 2 HTML5 Bitmap Gaming
3
Core
GUIMark 2 HTML5 Text Columns
3
Core
Asteroids HTML5 Canvas 2D And JavaScript2
Core
Psychedelic Browsing
2
Observation
Hardware Acceleration Stress Test
2
Dated
Mozilla WebGL FishIE
5
Observation
WebGL Solar System5
Observation
Efficiency Benchmarks (5)
Memory Usage: Single Tab
3
Core
Memory Usage: 40 Tabs
3
Core
Memory Management: -39 Tabs
3
Core
Memory Management: -39 Tabs (extra 2 minutes)
3
Core
Reliability Benchmarks (1)
Proper Page Loads
3
Core
Conformance Benchmarks (3)
Ecma test2621
Core
Peacekeeper 2.0 HTML5 Capabilities1
Core
HTML5Test.com1
Core


Legend

Throughout this article there are a ton of charts, many of which house data from both Windows 7 and Mac OS X. The data in these charts is sorted by the Windows 7 score, with the best performer on top and the worst at the bottom. Windows 7 scores are represented by blue bars and OS X scores are in green. If an OS X browser outperforms all of the Windows 7 browsers, we'll indicate this occurrence by changing the color of the winning OS X browser from green to red.

Now, let's fire off the starting pistol and let the race begin!

4. Startup Time Performance Benchmarks

Since Web Browser Grand Prix 2: The Top 5 Tested And Ranked, we've had two versions of the startup time tests: single tab and eight tabs. This time, we split the startup time testing into two more groups: cold and hot.

Cold Startup Time

Cold startup times test the amount of time a browser takes to open on a freshly rebooted system. This is how long you can expect to wait when first turning on your computer and opening a Web browser.

Single Tab

Google.com remains the test page we use for our single-tab startup benchmark.

All five Windows 7 Web browsers finish within 0.02 seconds of each other. By any normal standard, this is insignificant. But technically, Apple Safari takes the lead, followed by Mozilla Firefox, Google Chrome, Opera, and Microsoft Internet Explorer.

The story changes in Mac OS X, with Mozilla Firefox not only placing first, but also beating the times of every single Windows browser. Second place goes to Apple's own Safari, with Chrome 16 taking third place 0.05 seconds behind Safari. Opera for OS X places last, trailing behind Chrome by nearly a tenth of a second.

Eight Tabs

The home pages of Google, YouTube, Yahoo!, and The Huffington Post, along with product pages on Amazon, Craigslist, Wikipedia, and eBay provide the eight test pages for this portion of the startup time benchmark.

When opening eight tabs, Microsoft's own IE9 takes the lead in Windows 7 with a result just north of five seconds. Following one-half second behind is Chrome and Opera in a dead heat. Firefox places fourth at 6.5 seconds, and Safari takes last place at nearly 9.5 seconds.

The order is completely reversed in Mac OS X, with Apple's browser taking first place at just under 6.5 seconds. Firefox 9 places second at just under seven seconds, while Opera takes third at just over seven seconds. Google Chrome brings up the rear with one second per tab.

When it comes to cold startup times, Chrome and Opera show much better in Windows than OS X, while Firefox is about the same under either operating system. Safari scores much better on its native platform in the eight-tab test, but finishes nearly equal to its Windows port opening a single tab.

Hot Startup Time

Hot startup times test how long it takes a browser to open on a system in which that browser has already been opened and closed. This is the time you would expect to wait if you have previously closed your browser, but haven't rebooted the system.

Single Tab

Hot startup times are dramatically lower than what we just saw, and the placing remains the same for both Windows 7 and Lion. Safari manages to hold onto the lead, and IE9 jumps from last place up to second. Firefox finishes third, followed by Chrome and Opera. All of the browsers perform better in Windows 7 with the exception of Firefox 9, which finishes slightly faster in Mac OS X.

Eight Tabs

In our eight-tab test, Opera takes the lead in Windows, followed by IE9. Chrome places third, while Firefox takes fourth. Safari finishes at an astonishing twelve seconds (two seconds higher than its cold time).

Google Chrome barely edges out the Norwegian browser for first place in OS X, with Firefox close behind in third. Oddly, Apple's own browser places last, about a full second behind the rest. 

5. Page Load Time Performance Benchmarks

Like the startup time tests, our page load times have also split into two groups of testing: uncached and cached.

Uncached Page Load Time

Non-cached page load time is taken from browsers with a completely cleared cache and clean history. Non-cached page load time is indicative of the loading time for a page you've never been to before.

Windows 7 Detail View

Mac OS X Detail View

Uncached Page Load Time Composite

This chart displays the average time each Web browser takes to load all nine test Web sites.

When loading brand new pages in Windows 7, it's Google Chrome that takes the lead with an average of 1.5 seconds. Second place is shared by Safari and IE9 at 1.7 seconds. Firefox 9 takes third at just 1.76 seconds, and Opera places last at 1.87 seconds.

The finishing order changes slightly in Mac OS X, though Chrome 16 still manages to hang on to first place. Second place goes to Firefox 9, which again earns nearly the same score as in Windows 7. Apple's Safari takes third on its native platform, while Opera for OS X is the only Web browser to take longer than two seconds, placing last.

Cached Page Load Time

The cached page load time tests are performed with the test pages already fully loaded into the browsers cache and history. Cached page load time is indicative of the loading time you can expect for a page you regularly visit.

Windows 7 Detail View

Mac OS X Detail View

Cached Page Load Time Composite

When loading previously-visited Web pages, Google Chrome again takes the lead in Windows 7 at three-quarters of a second, followed by WebKit cousin Apple Safari at 0.8 seconds. Opera shoots up from fifth place to third for cached pages, loading in around one second. Firefox 9 remains in fourth place with a time of 1.15 seconds. IE9 drops to fifth place from third to finish at nearly 1.5 seconds.

In Mac OS X, Apple Safari barely edges out Chrome for first place (both place under the one-second mark). Firefox takes third at 1.1 seconds, and Opera finishes last at 1.25 seconds.

6. JavaScript Performance Benchmarks

FutureMark Peacekeeper

In Windows 7, Chrome tops the new version of Peacekeeper with a score of 3775 on the MacBook Air. Opera follows in second place by 650 points, with Safari another 1000 behind Opera in third. IE9 comes in fourth, just shy of 2000 points, with Firefox 9 trailing behind with a score just under 1700.

Chrome also claims victory in Mac OS X, though it falls 500 points lower than its Windows 7 score. Apple Safari earns a second-place finish on its native platform with a score of 2500. Opera for OS X takes third place just under 2300, while Mozilla Firefox again comes in last at just over 1600.

Mozilla Kraken v1.1

Google Kraken v1.1 Mod

Kraken v1.1 Composite

The differences between the original Mozilla version of Kraken and the Google modification are minute. We're considering dropping the Google Mod from the test suite and sticking with Mozilla's original Kraken test in future editions of the Web Browser Grand Prix. Sound off in the comments with your own opinion.

Apple SunSpider v0.9.1

IE9 still wins the original version of the SunSpider benchmark due to its "dead code elimination" optimizations. Firefox 9 places second, nearly 35 milliseconds behind IE9. Safari comes in third, followed by Chrome and Opera in a near-tie for last place.

The placing remains the same in OS X, though both Firefox and Safari perform better on Apple's OS, whereas Chrome and Opera are speedier on Microsoft's.

Google SunSpider v0.9.1 Mod

The Google Mod of SunSpider pushes Chrome from fourth place up to first in Windows 7, with Firefox closely in tow. IE9 moves from first to third, while Safari drops to fourth, and Opera retains its last-place spot.

Firefox 9 for OS X takes the lead on that platform and beats all of the Windows 7 scores. Chrome takes second place, followed by Safari in third, and Opera in fourth.

SunSpider v0.9.1 Composite

The composite SunSpider score puts Firefox in the lead for Windows, followed closely by IE9 in second place, with Chrome even closer behind in third. Safari places fourth and Opera solidly falls into last place. Firefox for OS X is again the big cross-platform winner, followed by Safari. Chrome takes third place on Apple's platform, while Opera cannot get out of last place on any version of this benchmark (and in any platform).

7. DOM And CSS Performance Benchmarks
DOM

Mozilla Dromeao DOM

As usual, Opera for Windows destroys the competition in the Dromeao DOM benchmark. Mozilla's own browser claims second place. Google earns a close third-place finish, while Apple Safari takes fourth, about 500 points behind Chrome. IE9 again places last by another 500 points behind Safari.

Firefox 9 takes the lead in Mac OS X, scoring even higher than its Windows version. Chrome takes second place and Safari takes third, which, like Firefox, also performs better than in Windows 7. Oddly enough, Opera for OS X places last, 1100 points lower than the Windows version!

CSS

Microsoft Maze Solver

Apple Safari earns first place in Microsoft's Maze Solver CSS benchmark, with the OS X version achieving the best cross-platform score. Google Chrome is right behind in second place with a difference of only 0.2-0.3 seconds. Opera takes third place by finishing in 11 seconds in Windows 7 and 20 in OS X. IE9 places fourth at just under 15 seconds. And Firefox performs horribly, taking well over one minute, placing last on both platforms.

8. Flash Performance Benchmarks

GUIMark 2 Flash

All of the Windows 7 browsers appear to have steady scores with no browser showing an abnormal weakness in any of the three HTML5 implementations.

Firefox and Chrome for OS X both seem to have an advantage in vector charting, while Firefox also shows a strength in text columns.

Once again, Opera, Safari, and IE9 all place close together at the top of the pile in this Windows 7-based test.

Opera places first with 41.5 FPS, followed by Safari with 40.5 FPS, and IE9 with 39.5 FPS. Firefox comes in fourth place at 35 FPS, while Chrome takes last place with 32 FPS.

Mozilla grabs the lead in OS X with 33 FPS, followed by Chrome with 32 FPS. Safari takes third at 30.5 FPS and Opera brings up the rear at 29.5 FPS. All of the browsers do worse in OS X than in Windows.

Flash Benchmark 2008

Safari takes the lead in Windows 7 by a decent margin over second-place finisher IE9. Opera pulls into third, followed by Firefox in fourth and Chrome in fifth.

The OS X placing changes significantly, as the last-place finisher in Windows 7, Firefox, takes the Apple gold. Safari takes a hit on its native platform to come in a very close second, though, trailed by Chrome (which also enjoys a benefit over the Windows version). Opera comes in last place, not far behind Chrome, but very far behind its Windows 7 score.

9. Java And Silverlight Performance Benchmarks
Java

GUIMark Java

Firefox and Chrome take the lead for Java performance in Windows 7, both earning close to 40 FPS. Opera earns a solid third-place finish at nearly 34 frames per second. Safari and IE9 bring up the rear at just over 32 FPS. All five Windows browsers score a perfectly playable 30+ FPS.

On the other hand, the four Mac OS X browsers only score in the 20 FPS range. Opera is the first-place finisher with a score that barely exceeds 20 FPS. Safari earns exactly 20 FPS to place second. Google Chrome comes in third at just under 19 FPS, and Firefox finishes last, right under 18 FPS.

Silverlight

Encog Silverlight

The Encog Silverlight Benchmark scores are close. But this time it's Opera that takes the lead in Windows. Firefox comes in second place, IE9 places third, Chrome earns fourth, and Safari winds up in fifth.

The placing changes significantly in Mac OS X, with Safari coming in first, followed by Opera, then Chrome, and then Firefox. The OS X scores are only about half of what we've seen in Windows. Remember, shorter bars are better here.

10. HTML5 Performance Benchmarks

Facebook JSGameBench 4.1

Firefox 9 running under Windows 7 is able to pull off another significant win in Facebook's JSGameBench test, followed by Internet Explorer. Chrome places a distant third with a sub-500 score. Safari fills fourth place at close to 250 points, while Opera trails far behind with about 150 points.

The tables turn in OS X, as Safari achieves the high score of nearly 1400 points. Chrome moves up to second place, earning about 50 more points than its Windows version. Firefox drops dramatically to third place with 266 points, a mere sixth of its showing in Windows 7. Opera again brings up the rear, not even earning 100 points.

GUIMark 2 HTML5

As a reminder, we use the single-pixel variant of the HTML vector charting test instead of the original two-pixel version due to unusually low scores achieved by WebKit-based browsers.

Windows 7 Detail View

Mac OS X Detail View

Cross-Platform Composite View

Firefox takes the lead in the Windows 7 GUIMark 2 HTML5 tests with a fantastic score of 47 FPS. IE9 earns second place at 44 FPS, while Chrome pulls in 37.5 FPS to take third. Safari and Opera both earn scores in the 28 FPS range to place fourth and fifth, respectively.

Safari is the first-place finisher in Mac OS X, achieving 45 FPS. Firefox places second with 28 FPS, and Opera is close behind with 27 FPS. Chrome brings up the rear with only 24.5 frames per second.

Asteroids HTML5 Canvas 2D

IE9 tops the Windows 7 charts in the Asteroids HTML5 Canvas 2D and JavaScript benchmark at 55 FPS. Safari and Chrome tie for second place with 47 FPS each. Opera places a respectable third, while Firefox again flails behind the pack, scoring only 21 FPS.

In OS X, it's Safari that earns the highest score with 54.5 FPS. Opera moves to second place with 38 FPS, and Chrome takes third with 35 FPS. Firefox remains in last place, achieving two frames per second more than its Windows showing. 

11. Harware Acceleration Performance Benchmarks

Microsoft Psychedelic Browsing

Internet Explorer still holds the highest score in Microsoft's own Psychedelic Browsing benchmark, followed by Firefox 9. Chrome takes third, Opera is in fourth, and the Windows version of Safari only earns 64.5 points for a last-place finish.

The scores are nearly reversed in OS X, with Safari earning a high score that's a few points higher than what Firefox demonstrated under Windows 7 to take its second-place finish. The rest of the browsers perform dismally in OS X. Opera comes in second just shy of 80 points. Firefox's 41.5 points are good enough for third place. And Chrome's 36 points earn it last place.

We still haven't received an explanation as to why the IE9 score jumped by several thousand points without the browser version being updated. If Microsoft's representatives ever respond to our inquiry we'll update the article. Until then, Psychedelic Browsing is under observation.

Mozilla Hardware Acceleration Stress Test

In Windows 7, IE9 and Firefox are again the only two Web browsers able to max out Mozilla's Hardware Acceleration Stress Test, sharing first place. Chrome falls to a distant second with 23 FPS, trailed closely by Opera's 22 FPS result. Safari only earns 13 FPS, placing it in last.

On Apple's platform, Safari takes the high score of 56 FPS. Opera places second with 19 FPS, followed by Chrome at 10 FPS, while Windows 7-winner Firefox only achieves 5 FPS on the Mac.

12. WebGL Performance Benchmarks

Mozilla WebGL FishIE Tank

We reduced the number of fish in this test to 2000 due to the more limited graphics hardware on the MacBook Air.

Firefox leads Chrome by 21 FPS in Windows 7, while Google holds a ten-frame lead over Mozilla in Mac OS X.

Strangely, Chrome did not display any fish in Mac OS X, though the frame rate fluctuated as if the test was working properly. While this benchmark puts Chrome in the lead on OS X, we have to consider this odd behavior in our final analysis.

Google WebGL Solar System

Firefox again has the edge over Chrome in Windows 7, this time by 5 FPS. Firefox manages to take the gold in Mac OS X, again leading Chrome by nearly five frames per second.

WebGL Composite

When both WebGL tests are averaged, Firefox has a clear lead in Windows 7, while Chrome has the advantage in Mac OS X by 1 FPS.

13. Memory Usage Efficiency Benchmarks

Light Load

IE9 uses the least amount of memory to display a single tab under Windows 7, just under 50 megabytes. Apple Safari comes in second place, using just over 50 megabytes, followed closely by Opera. Chrome places fourth, consuming nearly 65 megabytes, while Firefox uses the most memory (90 megabytes), placing last.

Safari leads in OS X at just under 60 megabytes, followed by Opera at just over 70. Firefox overtakes Chrome to place third by using 120 megabytes, while Google's browser uses the most memory (140 megabytes).

Heavy Load

Under a heavy load of 40 tabs, Safari for Windows uses the least amount of memory (just 725 MB). Firefox comes in second at 910 MB, followed closely by Opera at 925 and Chrome at 995. Microsoft's own IE9 uses the most memory (1.75 GB).

Opera is the only competitor to use less than a gigabyte of memory in Mac OS X. Safari comes in a close second, eating just over 1 GB, followed by Firefox at 1.25 GB. Chrome reports a whopping 2.3 GB of usage with 40 tabs open, which is significantly more than any other browser.

14. Memory Management Efficiency Benchmarks

Immediately After Closing 39 Tabs

After immediately closing 39 of the 40 tabs, Chrome for Windows drops down to 140 MB, which is just over twice its single-tab total. Opera places second at 415 MB, followed closely by Safari using 453 MB. IE9 only drops down to a half of a gigabyte, earning fourth place, while Firefox 9 retains more than 600 MB to place last.

In OS X, the order remains the same, though the totals are substantially higher than in Windows 7.

Two Minutes Later

Waiting an additional two minutes sees Internet Explorer drop down to only 57 MB, just 10 MB more than its single-tab total. The extra two minutes don't help Chrome's memory usage, although the browser retains second place. Firefox releases another 400 MB to Windows, taking third place. Safari lets go of another 100 MB to take fourth, and Opera stays at around 400 MB, finishing last.

In OS X, the additional two minutes does nothing to shake up the finishing order. Chrome still hovers around 140 MB and Opera hangs around 575 MB. Safari drops 200 MB to an ominous 666 MB and Firefox sheds another 300 MB to stabilize at just over 700.

Interestingly, throughout memory testing, the OS X-based browsers use a substantially greater amount of RAM than their Windows-based counterparts.

15. Page Load Reliability Benchmarks

In our Windows 7 proper page-load test, Firefox and Opera defend their reputations as rock-solid Web browsers, both displaying less than two failed elements in our 40-tab test, tying for first place. Safari comes in second, displaying fewer than four failed pages. And Chrome places third with just over five failures. That's a marked improvement over recent versions. IE9 again displays a serious problem when attempting to simultaneously render multiple Web pages, as more than half of its pages suffer missing or broken elements.

The order remains largely the same in Mac OS X, with Opera taking the lead by displaying only two failures. Firefox comes in a close second by averaging less than three failures. Chrome again displays around five failures, placing third. Surprisingly, Safari drops to last place by showing us nearly seven failures (more than its Windows version).

16. Standards Conformance Benchmarks

JavaScript

Ecma test262

Version 11.60 allows Opera to shoot from the bottom to the top of the pack with a near-perfect score of 11 107 in Ecma's test262. Firefox comes in a close second place by scoring 10 944, while arch-rival Internet Explorer is more than 200 points behind in third place. Chrome falls to fourth with 10 692 points, while Safari places last at 10 334 in Windows 7 and 10 335 in Mac OS X.

HTML5

HTML5Test.com

Chrome takes the lead in Windows 7 with a score of 344 out of 450, plus 13 bonus points. Opera comes in second place with a score of 325 and nine bonus points. Firefox achieves 314 and nine bonus points. Safari demonstrates 252 and two bonus points for fourth place, while IE9 is in last place with a score of only 141 points and five bonus.

Although the placing order remains the same in Mac OS X, Google Chrome earns one point less than in Windows, and Safari gains 41 regular points and six bonus points under its native platform.

FutureMark Peacekeeper 2.0 HTML5 Capabilities

FutureMark's new version of Peacekeeper also has an HTML5 conformance measurement, known as HTML5 Capabilities. The maximum score is seven, and Google Chrome is the only browser from either platform to earn a perfect score, sealing its victory. Firefox attains a six out of seven to place second, while Opera manages a score of five, putting it in third place. IE9 comes in fourth place with a score of three.

Although Safari for Windows only gets a single point, Safari for OS X earns four.

Google Chrome once again tops the conformance composite with the only A (minus) grade out of the bunch. Firefox 9 lands a solid B grade, while Opera shoots up to a B-. Both IE9 and the Windows version of Apple Safari land an embarrassing failing grade, although Safari manages a C- on OS X.

17. Benchmark Analysis

In our last installment, Web Browser Grand Prix 7: Firefox 7, Chrome 14, Opera 11.51, we completely dropped the raw placing tables in favor of the analysis tables that more closely reflect the scale of victory. Let's quickly take a moment to go over what these mean.

The analysis tables hold categories for each type of benchmark. For example, Mozilla Dromaeo DOM is represented by the DOM category, while Peacekeeper, the Krakens, and SunSpiders are represented together under the JavaScript category.

Each category has four columns: winner, strong, acceptable, and weak. Winner is obviously the browser that achieves the highest scores for the category. The strong column is for those browsers exhibiting superior performance, but no victory. Acceptable is for browsers that perform neither spectacularly nor poorly, but merely adequately. For tests that measure frame rates, a score near the 30 FPS range gets that browser filed into the acceptable column. The weak column is for browsers that perform poorly or substantially lower than their competitors.

In the event of a complete tie in the analysis tables, we simply go back to the individual benchmarks and look at the raw difference in scores.

The Windows 7-based standings for Chrome 16, Firefox 9, Internet Explorer 9, Opera 11.60, and Safari 5.1.2 are found in the table below.

Windows 7 Analysis Table


Winner
Strong
Acceptable
Weak
Performance Benchmarks
Startup Time, Light
SafariChrome, Firefox, Internet Explorer, Opera

Startup Time, Heavy
Internet ExplorerChrome, OperaFirefoxSafari
Page Load Time, Uncached
Chrome
Firefox, Internet Explorer, Opera, Safari


Page Load Time, Cached
Chrome
Safari
Firefox, Opera
Internet Explorer
JavaScriptChrome
Firefox
Opera
Internet Explorer, Safari
DOMOpera
Chrome, Firefox
Safari
Internet Explorer
CSSSafari
Chrome
Internet Explorer, Opera
Firefox
FlashSafari
Internet Explorer, Opera
Chrome, Firefox

JavaFirefox
Chrome
Internet Explorer, Opera, Safari

SilverlightOpera

Firefox, Internet ExplorerChrome, Safari
HTML5Internet Explorer
Firefox
Chrome, Safari
Opera
HTML5 Hardware AccelerationInternet Explorer
Firefox

Chrome, Opera, Safari
WebGLFirefox

Chrome
Internet Explorer, Opera, Safari
Efficiency Benchmarks
Memory Usage, Light
Internet Explorer
Safari, Opera
Chrome, Firefox

Memory Usage, Heavy
Safari

Chrome, Firefox, Opera
Internet Explorer
Memory Management
Chrome
Internet Explorer
Firefox
Opera, Safari
Reliability Benchmarks
Proper Page Loads
Firefox, Opera
Safari
Chrome
Internet Explorer
Conformance Benchmarks
HTML5
Chrome
Firefox, Opera

Internet Explorer, Safari
JavaScript
Opera
Firefox
Chrome, Internet Explorer, Safari


Now let's see how Chrome 16, Firefox 9, Opera 11.60, and Safari 5.1.2 fare on the MacBook Air's native platform of Mac OS X.

Mac OS X (Lion) Analysis Table


Winner
Strong
Acceptable
Weak
Performance Benchmarks
Startup Time, Light
Safari
Firefox
Chrome
Opera
Startup Time, Heavy
Firefox
Opera
Chrome, Safari

Page Load Time, Uncached
Chrome
Firefox
Safari
Opera
Page Load Time, Cached
Safari
Chrome
Firefox, Opera

JavaScriptChrome
Firefox
Safari
Opera
DOMFirefox
Chrome, Safari
Opera

CSSSafari
Chrome
Opera
Firefox
FlashFirefox

Chrome, Opera, Safari

JavaOpera

Safari
Chrome, Firefox
SilverlightSafari


Chrome, Firefox, Opera
HTML5Safari

Chrome, Opera
Firefox
HTML5 Hardware AccelerationSafari


Chrome, Firefox, Opera
WebGL

Chrome, Firefox
Opera, Safari
Efficiency Benchmarks
Memory Usage, Light
Safari

Opera
Firefox, Chrome
Memory Usage, Heavy
Opera
Safari
Firefox
Chrome
Memory Management
Chrome


Firefox, Opera, Safari
Reliability Benchmarks
Proper Page Loads
Opera
Firefox
Chrome, Safari

Conformance Benchmarks
HTML5
Chrome
Firefox, Opera
Safari

JavaScript
Opera
Firefox
Chrome, Safari


Without further ado, let's crown our Web Browser Grand Prix VIII champions.

18. Crowning Two Champions In Windows 7 And OS X

Web Browser Grand Prix Champion, The Windows Circuit

Although Firefox demonstrates the least number of wins (one less than IE9, Opera, and Safari, and two less than Chrome), it achieves the highest number of strong scores and only turns in one weak performance. Firefox 9 solidifies Mozilla's Web Browser Grand Prix championship, which was precariously handed out to Firefox 7 last September. Congratulations Mozilla! It's good to see that this browser war veteran still has some fight left in it.

Chrome 16 is still just a minor performance update away from matching or perhaps even surpassing Firefox 9 though, and the recent improvements to Firefox seem to have absolutely no effect on Google's march to the top of Web browser market share.

Opera is still hanging onto a strong third-place position with version 11.60, especially considering it still lacks HTML5 hardware acceleration and WebGL support. But Opera 12 is on the way, and it promises to seriously shake things up, just like the last two whole-number releases out of Norway.

Microsoft Internet Explorer has been stagnating for several months, and it's really starting to show. Last year, IE9 won two consecutive Web Browser Grand Prix championships and stayed near the top of the pack for some time afterward. The browser now resides at the bottom of the heap, tied for last place Apple's offering in Windows 7.

While the Windows version of Safari moved forward since our last look, the progress isn't enough to move the browser out of last place. In Windows 7, that is...

Web Browser Grand Prix Champion, The Mac OS X Circuit


Despite the best efforts of Google, Mozilla, and Opera, Apple Safari retains its OS X Web Browser Grand Prix crown. With a staggering number of wins and an equally impressive lack of weaknesses, Safari 5.1.2 simply owns on its native platform.

The OS X runner-up, on the other hand, is highly debatable. We're going call it in favor of incumbent number two, Google Chrome. Firefox 9 really pulled ahead from the last-place finish it received back in August, though.

Unfortunately, the Mac version of Opera is a mere shadow of its Windows version, landing the Norwegian powerhouse in last place.  

Operating System Comparison

The red bars that occasionally appear in our charts denote when an OS X-based browser beats all of the Windows 7-based competition. We use the word occasionally because we only had to switch the Mac OS X green bars to red four times. That's four out of 35 eligible charts, as opposed to the 10 out of 29 OS X earned on the Hackintosh system we used in Web Browser Grand Prix VI: Firefox 6, Chrome 13, Mac OS X Lion. While many Mac fans expected to see OS X really hammer Windows 7 on a genuine Mac, the home court advantage didn't do Apple any favors.

So, Redmond wins yet another Mac versus PC Web Browser Grand Prix, this time on a mobile system (and a brand new Core i7-based MacBook, to boot). Ouch.

Follow Adam on Twitter.