Skip to main content

Web Browser Grand Prix 9: Chrome 17, Firefox 10, And Ubuntu

WBGP9 Test Suite And Methodology

We restart the computer and allow it to idle for a few minutes before benchmarking each browser. Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variance. 

All tests are placed into one of four groups: core, observation, dated, and quarantine. Core tests are considered current. These tests are usually trusted industry standards or our own creations, and they make up the foundation of the WBGP suite. Tests that are either generally unknown, mostly untested, or just too bleeding-edge are placed under observation. Tests classified as dated are either outdated, losing relevance, or otherwise need replacing. We are actively seeking community feedback and contributions regarding alternatives to these benchmarks. The final group is for quarantined benchmarks. Benchmarks find their way into quarantine by delivering dubious results or by being gamed. Whenever benchmarks that test the same thing yield conflicting results, more weight is given to tests with a better rating when creating the analysis tables.

The table below lists all 51 of the tests currently in our suite (along with a version number and link, where applicable), number of iterations performed, and current rating:

Tom's Hardware Web Browser Grand Prix Test Suite v9.0
Test NameIterationsRating
Performance Tests (44)
Cold Startup Time: Single Tab3Core
Cold Startup Time: Eight Tabs3Core
Hot Startup Time: Single Tab3Core
Hot Startup Time: Eight Tabs3Core
Uncached Page Load Times (9 Test Pages)5Core
Cached Page Load Times (9 Test Pages)5Core
Kraken v1.12Core
Google SunSpider v0.91 Mod2Core
FutureMark Peacekeeper 2.02Core
Dromaeo DOM2Core
Maze Solver5Core
GUIMark2 Flash Vector Charting3Core
GUIMark2 Flash Bitmap Gaming3Core
GUIMark2 Flash Text Columns3Core
Flash Benchmark 2008 v1.09.12Core
GUIMark Java3Dated
Encog Silverlight3Dated
Facebook JSGameBench v4.12Core
GUIMark 2 HTML5 Vector Charting (1 pixel variant)3Core
GUIMark 2 HTML5 Bitmap Gaming3Core
GUIMark 2 HTML5 Text Columns3Core
Asteroids HTML5 Canvas 2D And JavaScript2Observation
Psychedelic Browsing2Core
Hardware Acceleration Stress Test2Dated
Mozilla WebGL FishIE5Core
WebGL Solar System5Observation
Efficiency Benchmarks (5)
Memory Usage: Single Tab3Core
Memory Usage: 40 Tabs3Core
Memory Management: -39 Tabs3Core
Memory Management: -39 Tabs (extra 2 minutes)3Core
Reliability Benchmarks (1)
Proper Page Loads3Core
Conformance Benchmarks (3)
Ecma test2621Core
Peacekeeper 2.0 HTML5 Capabilities1Core
HTML5Test.com1Core

Detailed individual methodologies are described on the pages corresponding to each benchmark.

Legend

There are a ton of charts throughout this article, many of which house data from both Windows 7 and Ubuntu 11.10. Data is sorted according to the Windows 7 score, with the best performer on top and the worst at the bottom. Windows 7 scores are represented by blue bars and Ubuntu 11.10 scores are in green. If an Ubuntu-based browser outperforms all of the Windows 7 browsers, we indicate this by changing the color of the winning Ubuntu browser from green to red.

  • mayankleoboy1
    just wondering if use of a DX11 capable GPU will change scores in some HTML5 and other benchmarks as the browsers use DX11 assisted rendering.

    Also, AMD driver support in linux is poor compared to Nvidia.
    For future Linux articles, can you use a Dx11 based Nvidia GPU?
    Reply
  • mayankleoboy1
    IMO, Firefox is concentrating more on HTML5, ignoring CSS and JavaScript.
    It does well in HTML5 benches but 99% of the websites use primarily CSS and JS and HTML3, in which Firefox does poorly.

    Reply
  • mayankleoboy1
    Waiting for OPERA12. It keeps impressing me.
    Even without hardware acceleration, it keeps up with the competition,

    When that beast launches, it will kill FF/IE and most probably chrome too.
    Reply
  • PreferLinux
    Who wants to guess that the poor Linux Flash and WebGL results were because Flash and WebGL don't use hardware acceleration with that graphics card and driver? I would be thinking so.
    Reply
  • mitch074
    Firefox performance took a dive starting with version 4, where all hardware acceleration was disabled: before then, in version 3.6, XRENDER was used when available (it was 4/5th as fast as IE9 on the same PC) while it is now really slow - it's all software.

    Moreover, the only driver enabled for hardware acceleration on Linux is the Nvidia driver: according to Mozilla (and verified by yours truly on AMD and Intel hardware), most display drivers in Linux suck when it comes to 2D rendering - ouch. Note that Mozilla and Google could add shims to circumvent those bugs, but they don't -not worth the effort, especially when driver makers could fix their bugs rather easily, leaving the browsers broken yet again.
    Reply
  • indian-art
    I use Chrome (19.0.1041.0 dev presently) the most on Linux (Ubuntu) and empirically I felt Chrome works very well. Now your tests confirm it.

    I find Opera 12 really nice too. It can run with Opera 11.61. Opera 12 has a silver icon & 11.61 has its classic red. I like Firefox & Epiphany too.

    Its a shame Safari and IE are not truly cross-platform.
    Reply
  • mayankleoboy1
    how many of those top 40 sites use HTML5?

    i think that the HTML5 scores should be weighed by a factor of the percent of top40 sites that use HTML5.
    This way actual importance of HTML5 can be judged in real world.
    Reply
  • nd22
    It's a shame Apple does not pay enough attention to the Windows market and optimize their browser! On Mac Safari is king of the hill - personal opinion of course!
    On Windows I feel that IE9 works really well for me, although Chrome is the speed demon! FF 4+ lost their appeal for me.
    Reply
  • forestie
    The OSes that are used are 64 bits but the browsers are mostly (all?) 32bits on Windows, and probably 64bits on Linux.

    Internet Explorer has 64bits builds on Win7, and Firefox has "almost" a 64bits browser on Windows too: Waterfox, which is a semi-official Firefox for 64bits Windows. Waterfox in particular claims huge improvements over base 32bits install, I would like to see how that translates into real-world.

    Not sure about availability of 64bits editions of other browsers on Windows.

    Here are my wishes:
    -clearly mention if the 32bits or 64bits version of the browser is used
    -where applicable and relevant, test with both 32bits and 64bits variants. I would like to see IE and FF split into 32 and 64 variants on Win for example.

    I personally migrated from FF to WF on my machines 3 weeks ago and find it noticeably faster in everyday use. WF is now my main browser.
    Reply
  • doive1231
    As long as phones keep using Android, Chrome will be the most popular browser for a long while. Google have got it all sorted.
    Reply