Web Browser Grand Prix: Firefox 15, Safari 6, OS X Mountain Lion

Round-ups
By
published

Today we're breaking out the Hackintosh for our first-ever Web Browser Grand Prix on Apple OS X 10.8 (Mountain Lion). How will Chrome 21, Firefox 15, Opera 12.02, and Safari 6 stack up against each other, and to IE9 and the rest of the Windows 7 browsers?

Test Analysis

Finishes

Each category of testing has four columns: Winner, Strong, Average, and Weak. The Winner is obviously the browser that achieves the highest scores in that category. The Strong column is for browsers exhibiting superior performance, but not achieving a first-place victory. Average is for browsers that perform adequately or in-line with a majority of their competitors. A Weak finish is assigned to browsers that perform poorly, or substantially lower than their competitors.

Brackets

In order to reflect how each category of testing affects the average end-user Web browsing experience, we need to create brackets (or levels of importance) to place the different categories of testing into.

Swipe to scroll horizontally
EssentialCSS, DOM, JavaScript, Reliability, Standards Conformance
ImportantFlash, HTML5, Memory Efficiency, Page Load Time, Responsiveness, Security, Startup Time
NonessentialJava, Silverlight
UnimportantHTML5 Hardware Acceleration, WebGL

The Essential bracket contains those categories of testing that are indispensable to rendering the vast majority of Web pages online today. The Important bracket is for categories not quite essential to browsing the Web, yet still affect the user experience to a great degree. The Nonessential bracket contains the popular plug-ins Java and Silverlight. While these plug-in technologies are nowhere near as ubiquitous as Flash, certain applications like corporate intranet apps and Netflix simply will not work without them. Finally, the Unimportant bracket is for emerging technologies, such as HTML5 Hardware Acceleration and WebGL, which still don't really exist outside of testing/demo sites.

Points

Now that the brackets are all sorted out, we can apply a numerical point system to the finishes of each bracket.

Swipe to scroll horizontally
Header Cell - Column 0 WinnerStrongAverageWeak
Essential2.521.5-2
Important21.51-1.5
Nonessential1.510.5-1
Unimportant10.50-0.5

As you can see, we decided to apply negative point values to the Weak finishes and start the Average performances at zero for the Unimportant bracket. The Winner has also been de-emphasized over Strong finishes, with just a small tie-breaking bonus going to Winner.

OS X 10.8 Analysis Table

Swipe to scroll horizontally
Header Cell - Column 0 WinnerStrongAverageWeak
Essential
CSSFirefoxRow 1 - Cell 2 Chrome, SafariOpera
DOMFirefoxRow 2 - Cell 2 Chrome, SafariOpera
JavaScriptChromeSafariFirefox, OperaRow 3 - Cell 4
ReliabilityOperaRow 4 - Cell 2 Firefox, SafariChrome
Standards ConformanceChromeRow 5 - Cell 2 Firefox, Opera, SafariRow 5 - Cell 4
Important
FlashFirefox, Opera, SafariChromeRow 7 - Cell 3 Row 7 - Cell 4
HTML5SafariRow 8 - Cell 2 ChromeFirefox, Opera
Memory EfficiencyChromeRow 9 - Cell 2 Firefox, Opera, SafariRow 9 - Cell 4
Page Load TimeChromeRow 10 - Cell 2 Firefox, Opera, SafariRow 10 - Cell 4
ResponsivenessOperaFirefox, SafariChromeRow 11 - Cell 4
SecurityChromeSafariFirefoxOpera
Startup TimeChromeSafariFirefoxOpera
Nonessential
JavaChrome, Firefox, Opera, SafariRow 15 - Cell 2 Row 15 - Cell 3 Row 15 - Cell 4
SilverlightChrome, Firefox, Opera, SafariRow 16 - Cell 2 Row 16 - Cell 3 Row 16 - Cell 4
Unimportant
HTML5 Hardware AccelerationSafariRow 18 - Cell 2 ChromeFirefox, Opera
WebGLFirefoxRow 19 - Cell 2 ChromeOpera, Safari

Now, let's see how the Windows 7 Web browsers compare to each other.

Windows 7 Analysis Table

Swipe to scroll horizontally
Header Cell - Column 0 WinnerStrongAverageWeak
Essential
CSSFirefoxRow 1 - Cell 2 Chrome, IEOpera
DOMChromeRow 2 - Cell 2 Firefox, IEOpera
JavaScriptChromeRow 3 - Cell 2 Firefox, OperaIE
ReliabilityOperaRow 4 - Cell 2 Chrome, Firefox, IERow 4 - Cell 4
Standards ConformanceChromeRow 5 - Cell 2 Firefox, OperaIE
Important
FlashFirefox, IE, OperaChromeRow 7 - Cell 3 Row 7 - Cell 4
HTML5IERow 8 - Cell 2 Chrome, FirefoxOpera
Memory EfficiencyChromeIE, FirefoxOperaRow 9 - Cell 4
Page Load TimeIERow 10 - Cell 2 Chrome, Firefox, OperaRow 10 - Cell 4
ResponsivenessOperaFirefoxChromeIE
SecurityChromeRow 12 - Cell 2 Firefox, IEOpera
Startup TimeChromeOperaFirefox, IERow 13 - Cell 4
Nonessential
JavaChrome, Firefox, IE, OperaRow 15 - Cell 2 Row 15 - Cell 3 Row 15 - Cell 4
SilverlightFirefoxChrome, IE, OperaRow 16 - Cell 3 Row 16 - Cell 4
Unimportant
HTML5 Hardware AccelerationFirefoxIEChromeOpera
WebGLFirefoxRow 19 - Cell 2 ChromeIE, Opera

And the winners are...

Adam Overa
57 Comments Comment from the forums
  • glurg
    chrome ftw
    Reply
  • Eggrenade
    It would be nice if I could view the additional charts with only one click, and not in a separate window.
    Reply
  • lahawzel
    It's nice to see Chrome performing so well, but I'm still waiting on the Chrome equivalents of all the plugins I use in FF before I think about switching. The web just doesn't feel the same without them.

    (The nice popular ones like ABP, Lazarus, Greasemonkey all have equivalents; some lesser-used plugins like Rikaichan also have ports by now. Only a matter of time!)
    Reply
  • bennaye
    chrome is absolutely deserving of the award. say what you will about the frequent patch releases touted as upgrades, chrome is a very good browser, as shown by this month's article. even on OSX there is only a small margin separating chrome and safari. but the one qualm i do have with chrome is the lack of add-ons compared to firefox. and i a lot of people share this concern. the add-ons do make the experience that much better.

    as always, a great read.
    Reply
  • Would like to see this again after IE10 is released.
    Reply
  • JOSHSKORN
    How about 64-bit Internet Explorer 9 vs Waterfox 15.0?
    Reply
  • adamovera
    bennayechrome is absolutely deserving of the award. say what you will about the frequent patch releases touted as upgrades, chrome is a very good browser, as shown by this month's article. even on OSX there is only a small margin separating chrome and safari. but the one qualm i do have with chrome is the lack of add-ons compared to firefox. and i a lot of people share this concern. the add-ons do make the experience that much better.as always, a great read.All versions of Chrome hold up incredibly well cross-platform, if you look back at the two Linux WBGPs, it won there, too. Thanks for reading!
    Reply
  • adamovera
    AdamsTaiwanWould like to see this again after IE10 is released.Absolutely, a Windows 8-based WBGP is already in the cards for October.
    Reply
  • adamovera
    JOSHSKORNHow about 64-bit Internet Explorer 9 vs Waterfox 15.0?When we have more stable 64-bit browsers, I'll definitely do a 64-bit WBGP - including versus their 32-bit counterparts.
    Reply
  • I wish Tom's would fiddle around with the settings of these browsers for these tests. In every System Builder Marathon you overclock the builds, why not try and crank the most speed while ensuring better memory management out of the browser as well?

    Testing these browsers at stock doesn't reveal even an eighth of the picture.
    Reply