Test Analysis
Finishes
Each category of testing has four columns: Winner, Strong, Average, and Weak. The Winner is obviously the browser that achieves the highest scores in that category. The Strong column is for browsers exhibiting superior performance, but not achieving a first-place victory. Average is for browsers that perform adequately or in-line with a majority of their competitors. A Weak finish is assigned to browsers that perform poorly, or substantially lower than their competitors.
Brackets
In order to reflect how each category of testing affects the average end-user Web browsing experience, we need to create brackets (or levels of importance) to place the different categories of testing into.
Essential | CSS, DOM, JavaScript, Reliability, Standards Conformance |
---|---|
Important | Flash, HTML5, Memory Efficiency, Page Load Time, Responsiveness, Security, Startup Time |
Nonessential | Java, Silverlight |
Unimportant | HTML5 Hardware Acceleration, WebGL |
The Essential bracket contains those categories of testing that are indispensable to rendering the vast majority of Web pages online today. The Important bracket is for categories not quite essential to browsing the Web, yet still affect the user experience to a great degree. The Nonessential bracket contains the popular plug-ins Java and Silverlight. While these plug-in technologies are nowhere near as ubiquitous as Flash, certain applications like corporate intranet apps and Netflix simply will not work without them. Finally, the Unimportant bracket is for emerging technologies, such as HTML5 Hardware Acceleration and WebGL, which still don't really exist outside of testing/demo sites.
Points
Now that the brackets are all sorted out, we can apply a numerical point system to the finishes of each bracket.
Header Cell - Column 0 | Winner | Strong | Average | Weak |
---|---|---|---|---|
Essential | 2.5 | 2 | 1.5 | -2 |
Important | 2 | 1.5 | 1 | -1.5 |
Nonessential | 1.5 | 1 | 0.5 | -1 |
Unimportant | 1 | 0.5 | 0 | -0.5 |
As you can see, we decided to apply negative point values to the Weak finishes and start the Average performances at zero for the Unimportant bracket. The Winner has also been de-emphasized over Strong finishes, with just a small tie-breaking bonus going to Winner.
OS X 10.8 Analysis Table
Header Cell - Column 0 | Winner | Strong | Average | Weak |
---|---|---|---|---|
Essential | ||||
CSS | Firefox | Row 1 - Cell 2 | Chrome, Safari | Opera |
DOM | Firefox | Row 2 - Cell 2 | Chrome, Safari | Opera |
JavaScript | Chrome | Safari | Firefox, Opera | Row 3 - Cell 4 |
Reliability | Opera | Row 4 - Cell 2 | Firefox, Safari | Chrome |
Standards Conformance | Chrome | Row 5 - Cell 2 | Firefox, Opera, Safari | Row 5 - Cell 4 |
Important | ||||
Flash | Firefox, Opera, Safari | Chrome | Row 7 - Cell 3 | Row 7 - Cell 4 |
HTML5 | Safari | Row 8 - Cell 2 | Chrome | Firefox, Opera |
Memory Efficiency | Chrome | Row 9 - Cell 2 | Firefox, Opera, Safari | Row 9 - Cell 4 |
Page Load Time | Chrome | Row 10 - Cell 2 | Firefox, Opera, Safari | Row 10 - Cell 4 |
Responsiveness | Opera | Firefox, Safari | Chrome | Row 11 - Cell 4 |
Security | Chrome | Safari | Firefox | Opera |
Startup Time | Chrome | Safari | Firefox | Opera |
Nonessential | ||||
Java | Chrome, Firefox, Opera, Safari | Row 15 - Cell 2 | Row 15 - Cell 3 | Row 15 - Cell 4 |
Silverlight | Chrome, Firefox, Opera, Safari | Row 16 - Cell 2 | Row 16 - Cell 3 | Row 16 - Cell 4 |
Unimportant | ||||
HTML5 Hardware Acceleration | Safari | Row 18 - Cell 2 | Chrome | Firefox, Opera |
WebGL | Firefox | Row 19 - Cell 2 | Chrome | Opera, Safari |
Now, let's see how the Windows 7 Web browsers compare to each other.
Windows 7 Analysis Table
Header Cell - Column 0 | Winner | Strong | Average | Weak |
---|---|---|---|---|
Essential | ||||
CSS | Firefox | Row 1 - Cell 2 | Chrome, IE | Opera |
DOM | Chrome | Row 2 - Cell 2 | Firefox, IE | Opera |
JavaScript | Chrome | Row 3 - Cell 2 | Firefox, Opera | IE |
Reliability | Opera | Row 4 - Cell 2 | Chrome, Firefox, IE | Row 4 - Cell 4 |
Standards Conformance | Chrome | Row 5 - Cell 2 | Firefox, Opera | IE |
Important | ||||
Flash | Firefox, IE, Opera | Chrome | Row 7 - Cell 3 | Row 7 - Cell 4 |
HTML5 | IE | Row 8 - Cell 2 | Chrome, Firefox | Opera |
Memory Efficiency | Chrome | IE, Firefox | Opera | Row 9 - Cell 4 |
Page Load Time | IE | Row 10 - Cell 2 | Chrome, Firefox, Opera | Row 10 - Cell 4 |
Responsiveness | Opera | Firefox | Chrome | IE |
Security | Chrome | Row 12 - Cell 2 | Firefox, IE | Opera |
Startup Time | Chrome | Opera | Firefox, IE | Row 13 - Cell 4 |
Nonessential | ||||
Java | Chrome, Firefox, IE, Opera | Row 15 - Cell 2 | Row 15 - Cell 3 | Row 15 - Cell 4 |
Silverlight | Firefox | Chrome, IE, Opera | Row 16 - Cell 3 | Row 16 - Cell 4 |
Unimportant | ||||
HTML5 Hardware Acceleration | Firefox | IE | Chrome | Opera |
WebGL | Firefox | Row 19 - Cell 2 | Chrome | IE, Opera |
And the winners are...