Sign in with
Sign up | Sign in

Test Analysis

Web Browser Grand Prix: Firefox 15, Safari 6, OS X Mountain Lion
By

Finishes

Each category of testing has four columns: Winner, Strong, Average, and Weak. The Winner is obviously the browser that achieves the highest scores in that category. The Strong column is for browsers exhibiting superior performance, but not achieving a first-place victory. Average is for browsers that perform adequately or in-line with a majority of their competitors. A Weak finish is assigned to browsers that perform poorly, or substantially lower than their competitors.

Brackets

In order to reflect how each category of testing affects the average end-user Web browsing experience, we need to create brackets (or levels of importance) to place the different categories of testing into.

Essential
CSS, DOM, JavaScript, Reliability, Standards Conformance
Important
Flash, HTML5, Memory Efficiency, Page Load Time, Responsiveness, Security, Startup Time
Nonessential
Java, Silverlight
Unimportant
HTML5 Hardware Acceleration, WebGL


The Essential bracket contains those categories of testing that are indispensable to rendering the vast majority of Web pages online today. The Important bracket is for categories not quite essential to browsing the Web, yet still affect the user experience to a great degree. The Nonessential bracket contains the popular plug-ins Java and Silverlight. While these plug-in technologies are nowhere near as ubiquitous as Flash, certain applications like corporate intranet apps and Netflix simply will not work without them. Finally, the Unimportant bracket is for emerging technologies, such as HTML5 Hardware Acceleration and WebGL, which still don't really exist outside of testing/demo sites.

Points

Now that the brackets are all sorted out, we can apply a numerical point system to the finishes of each bracket.


Winner
Strong
Average
Weak
Essential
2.5
2
1.5
-2
Important
2
1.5
1
-1.5
Nonessential
1.5
1
0.5
-1
Unimportant
1
0.5
0
-0.5


As you can see, we decided to apply negative point values to the Weak finishes and start the Average performances at zero for the Unimportant bracket. The Winner has also been de-emphasized over Strong finishes, with just a small tie-breaking bonus going to Winner.

OS X 10.8 Analysis Table


Winner
Strong
Average
Weak
Essential
CSS
Firefox

Chrome, Safari
Opera
DOM
Firefox

Chrome, Safari
Opera
JavaScript
Chrome
Safari
Firefox, Opera

Reliability
Opera

Firefox, Safari
Chrome
Standards Conformance
Chrome

Firefox, Opera, Safari

Important
Flash
Firefox, Opera, Safari
Chrome


HTML5
Safari

Chrome
Firefox, Opera
Memory Efficiency
Chrome

Firefox, Opera, Safari

Page Load Time
Chrome

Firefox, Opera, Safari

Responsiveness
Opera
Firefox, Safari
Chrome

Security
Chrome
Safari
Firefox
Opera
Startup Time
Chrome
Safari
Firefox
Opera
Nonessential
Java
Chrome, Firefox, Opera, Safari



Silverlight
Chrome, Firefox, Opera, Safari


Unimportant
HTML5 Hardware Acceleration
Safari

Chrome
Firefox, Opera
WebGL
Firefox

Chrome
Opera, Safari


Now, let's see how the Windows 7 Web browsers compare to each other.

Windows 7 Analysis Table


Winner
Strong
Average
Weak
Essential
CSS
Firefox

Chrome, IE
Opera
DOM
Chrome

Firefox, IE
Opera
JavaScript
Chrome

Firefox, Opera
IE
Reliability
Opera

Chrome, Firefox, IE

Standards Conformance
Chrome

Firefox, Opera
IE
Important
Flash
Firefox, IE, Opera
Chrome


HTML5
IE

Chrome, Firefox
Opera
Memory Efficiency
Chrome
IE, Firefox
Opera

Page Load Time
IE

Chrome, Firefox, Opera

Responsiveness
Opera
Firefox
Chrome
IE
Security
Chrome

Firefox, IE
Opera
Startup Time
Chrome
Opera
Firefox, IE

Nonessential
Java
Chrome, Firefox, IE, Opera



Silverlight
Firefox
Chrome, IE, Opera


Unimportant
HTML5 Hardware Acceleration
Firefox
IE
Chrome
Opera
WebGL
Firefox

Chrome
IE, Opera


And the winners are...

React To This Article