Sign in with
Sign up | Sign in

Test Setup And Benchmark Suite

Which Web Browser Is Best Under Windows 8?
By

With the exception of a new cable modem, our test setup hasn't changed since the last time we ran the Grand Prix.

Test Setup And Benchmark Suite

Test System Specs
Operating System 1
Microsoft Windows 8 Enterprise Evaluation (64-bit)
Wiindows 8 DesktopWiindows 8 Desktop
Operating System 2
Microsoft Windows 7 Ultimate SP1 (64-bit)
Windows 7 DesktopWindows 7 Desktop
Processor
Intel Core i5-2500K @ 3.3 GHz (quad-core)
Motherboard
Gigabyte GA-Z68XP-UD3 (F10 BIOS)
Memory
8 GB Crucial DDR3 @ 1333 MT/s (2 x 4 GB)
Graphics
Asus GeForce GTX 560 Ti 1 GB GDDR5 (PCIe 2.0 x16)
Storage
Seagate Barracuda 7200.12 500 GB SATA 3Gb/s, 7200 RPM, 16 MB Cache
Optical
Asus DRW-24B1ST/BLK/B/AS
Power Supply
Corsair TX750W (750 W max)
Case
Zalman MS-1000 HS2
CPU Cooler
Scythe Mugen 2 Revision B
Monitor
HP 2509b 25-inch LCD (1920x1080)
Keyboard
Logitech Wireless Keyboard K320
Mouse
Logitech Wireless Trackball M570
Local Web Server Specs
Operating System
Ubuntu 12.04.1 LTS Server Edition "Precise Pangolin" (32-bit)
Processor
Intel Pentium 4 @ 2.41 GHz
Motherboard
Biostar P4M80-M4
Memory
768 MB DDR @ 333 MT/s
Graphics
Nvidia GeForce FX 5500 128 MB DDR (AGP)
Storage
Western Digital Caviar SE WD1600AAJD, 160 GB EIDE, 7200 RPM
Extra Packages
Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH, Node.js, NPM
Network Specs
ISP Service
Cox Preferred (18 Mb/s down, 2 Mb/s up)
Modem
Arris Touchstone Telephony Modem TM502G
Router
Linksys WRT54G2 V1
Benchmark Suite
Page Load Time
Cold Start Time (Yahoo! homepage, Cached)
Hot Start Time (Yahoo! homepage, Cached)
Cold Start Time (Eight Tabs, Cached)
Hot Start Time (Eight Tabs, Cached)
Uncached Load Time (Eight Test Pages)
Cached Load Time (Eight Test Pages)
HTML5
GUIMark2 HTML5 (Four Tests)
Asteroids HTML5 Canvas 2D And JavaScript
HTML5 Canvas Performance Test
Impact HTML5 Benchmark
CSS
CSS Stress Test And Performance Profiling - CSS Speed Test
CSS Stress Test And Performance Profiling - CSS3 Speed Test
Maze Solver
JavaScript
RIABench JavaScript (Eight Tests)
Apple SunSpider v0.9.1 (Google Mod)
Mozilla Kraken v1.1
Futuremark Peacekeeper v2.0
DOM
Mozilla Dromaeo DOM Core
Hardware AccelerationFacebook JSGameBench v0.4.1
WebVizBench
Psychedelic Browsing
Mozilla WebGL FishIE Tank
Chrome Experiments WebGL Solar System
Memory Efficiency
Memory Usage (Single Tab)
Memory Usage (40 Tabs)
Memory Management (-39 Tabs)
Memory Management (-39 Tabs, Two Additional Minutes)
Reliability
Proper Page Loads
SecurityBrowserscope Security
Standards ConformanceHTML5Test.com
The CSS3 Test
Ecmascript Language test262


Although applicable links are included in the table above, we've also created a delicious account dedicated to chronicling links to Web Browser Grand Prix benchmarks. We removed the Responsiveness measurement because they're too subjective and too difficult to properly gauge on the modern test system. Detailed methodologies are explained on the individual benchmark pages.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 89 comments.
This thread is closed for comments
Top Comments
  • 16 Hide
    jupiter optimus maximus , November 19, 2012 4:37 AM
    Quote:
    And we're also passing the torch from Windows 7 to Windows 8.

    We are going to miss you on Web Browser Grand Prix, Windows 7
  • 12 Hide
    mayankleoboy1 , November 19, 2012 3:59 AM
    Any technical reason why browser performs generally better in Win8 ? Even the 'WHQL' drivers from Nvidia and AMD arent quite mature for Win8.
    Games and applications did not show any improvement in Win8 over Win7.
  • 11 Hide
    randomizer , November 19, 2012 7:08 AM
    mayankleoboy11. IMHO, enabling these settings would have made Opera more competitive and this article fairer.


    Running every browser with its default configuration except one is not a fair test.
Other Comments
  • -8 Hide
    mayankleoboy1 , November 19, 2012 3:57 AM
    1. Did you ensure that Opera has Hardware acceleration and WebGL enabled in about:config ? AFAIk, Opera does not enable HWA by default.

    2. I find the over-reliance on "Internet Explorer Test drive" benchmarks disturbing. Most use code that is inefficient and not used anywhere else on the web, making it quite theoretical.

    3. +1 for using Google Octane benchmark. Both google and mozilla agree that this is a good real-world benchmark.

    4. Addition of the "Maze solver" benchmark is disappointing.

    5. Why remove the subjective smoothness ? 95% of the time, subjective smoothness is what lures a person to use a specific browser. People use a browser, not run benchmarks on it all day. Subjectively, no browser can beat Google Chrome. Then comes Opera , Firefox and far lastly, IE10.
  • 12 Hide
    mayankleoboy1 , November 19, 2012 3:59 AM
    Any technical reason why browser performs generally better in Win8 ? Even the 'WHQL' drivers from Nvidia and AMD arent quite mature for Win8.
    Games and applications did not show any improvement in Win8 over Win7.
  • 10 Hide
    adamovera , November 19, 2012 4:23 AM
    mayankleoboy11. Did you ensure that Opera has Hardware acceleration and WebGL enabled in about:config ? AFAIk, Opera does not enable HWA by default.2. I find the over-reliance on "Internet Explorer Test drive" benchmarks disturbing. Most use code that is inefficient and not used anywhere else on the web, making it quite theoretical.3. +1 for using Google Octane benchmark. Both google and mozilla agree that this is a good real-world benchmark.4. Addition of the "Maze solver" benchmark is disappointing.5. Why remove the subjective smoothness ? 95% of the time, subjective smoothness is what lures a person to use a specific browser. People use a browser, not run benchmarks on it all day. Subjectively, no browser can beat Google Chrome. Then comes Opera , Firefox and far lastly, IE10.

    1) We use fresh installs at default settings; Opera does not enable HWA by default.
    2) The only IETestDrive tests we use are Psychedelic Browsing and Maze Solver, and IE regularly loses to competitors on both.
    3) Octane was not used because it had issues with IE9 and Opera 12.10.
    4) We definitely need a new CSS test, but the only other options are outdated or on IETestDrive - unfortunately, Kaizoumark doesn't work with IE10.
    5) It's really difficult to see that kind of stuff on a modern test system, but I will say that Chrome and IE10 are about equal in that department, with Firefox and Opera noticeably more choppy right at the beginning of the 40-tab load.
  • 7 Hide
    adamovera , November 19, 2012 4:34 AM
    mayankleoboy1Any technical reason why browser performs generally better in Win8 ? Even the 'WHQL' drivers from Nvidia and AMD arent quite mature for Win8.Games and applications did not show any improvement in Win8 over Win7.

    Not sure, the Nvidia drivers used were the same version on both OSes.
  • 16 Hide
    jupiter optimus maximus , November 19, 2012 4:37 AM
    Quote:
    And we're also passing the torch from Windows 7 to Windows 8.

    We are going to miss you on Web Browser Grand Prix, Windows 7
  • 2 Hide
    mayankleoboy1 , November 19, 2012 4:44 AM
    Quote:
    1) We use fresh installs at default settings; Opera does not enable HWA by default.
    2) The only IETestDrive tests we use are Psychedelic Browsing and Maze Solver, and IE regularly loses to competitors on both.
    3) Octane was not used because it had issues with IE9 and Opera 12.10.
    4) We definitely need a new CSS test, but the only other options are outdated or on IETestDrive - unfortunately, Kaizoumark doesn't work with IE10.
    5) It's really difficult to see that kind of stuff on a modern test system, but I will say that Chrome and IE10 are about equal in that department, with Firefox and Opera noticeably more choppy right at the beginning of the 40-tab load.



    1. IMHO, enabling these settings would have made Opera more competitive and this article fairer.

    3. Whoops, misread that. But this is a good benchmark. Robohornet and robohornet pro are complete jokes.
    4. Just exclude the maze solver. Its bad coding, as any web developer can tell you.
    5. Thats exactly what i'm saying. This needs to be factored in the overall score. You want the browser UI to always remain smooth. UI choppiness is unacceptable and sloppy coding. We are not living in the 90's anymore.


    The one thing i dislike in Chrome is the memory bloat when opening many tabs. In the 40tab test, FF uses 600 MB. Chrome uses 1600MB :o . That is probably an iverhead of using separate processes for each tab. That is excellent for smoothness and UI fluidity. But shameful for memory consumption. I guess devs need to find a middle path.
  • 5 Hide
    mayankleoboy1 , November 19, 2012 4:57 AM
    Both 'mozilla kraken' and 'Google sunspider' benchmarks need to be retired . They are old, and all the major browsers have optimizations to score better on them.
    Plus, they heavily test features that are not used anywhere else on teh web.

    Example : Sunspider makes a billion manipulations to the the "date" variable. Mozilla did not have any optimization for this. So it scored poorly on Sunspider. After numerous 'review sites' started using sunspider to test FF Vs Chrome, mozilla developers had to reluctantly add the same optimisation (which is basically a separate buffer to store the date). Of course, nowhere on the web is the date variable used in this manner. So its optimization for an artificial test.
  • 1 Hide
    wilem_WAR246810 , November 19, 2012 5:15 AM
    "The King Is Dead, Long Live The King!" am I the only one who thought of Megadeth?
  • 3 Hide
    deepblue08 , November 19, 2012 5:18 AM
    mayankleoboy1Any technical reason why browser performs generally better in Win8 ? Even the 'WHQL' drivers from Nvidia and AMD arent quite mature for Win8.Games and applications did not show any improvement in Win8 over Win7.


    As far as I heard there are significant under-the-hood improvements in Win8, in terms of memory efficiency and multi-core usage.
  • 4 Hide
    epileptic , November 19, 2012 5:38 AM
    Is it Opera x64 or x86? I remember having tested Opera 12 and the startup was very slow. I'm still using 11.64 atm. The only thing keeping me from moving to Firefox is how sluggish the UI feels... I'd also have to find a new mail client. :/ 
  • 2 Hide
    adamovera , November 19, 2012 5:44 AM
    mayankleoboy11. IMHO, enabling these settings would have made Opera more competitive and this article fairer.3. Whoops, misread that. But this is a good benchmark. Robohornet and robohornet pro are complete jokes.4. Just exclude the maze solver. Its bad coding, as any web developer can tell you.5. Thats exactly what i'm saying. This needs to be factored in the overall score. You want the browser UI to always remain smooth. UI choppiness is unacceptable and sloppy coding. We are not living in the 90's anymore. The one thing i dislike in Chrome is the memory bloat when opening many tabs. In the 40tab test, FF uses 600 MB. Chrome uses 1600MB . That is probably an iverhead of using separate processes for each tab. That is excellent for smoothness and UI fluidity. But shameful for memory consumption. I guess devs need to find a middle path.

    Chrome scales quite nicely to the available memory, it doesn't need the 1.6 GB to display all those tabs, it just uses that much on a system with tons of spare memory. We saw that Chrome can get by with just 450 MB on an XP-based beige box with only 768 MB total system memory.
  • 6 Hide
    adamovera , November 19, 2012 5:47 AM
    wilem_WAR246810"The King Is Dead, Long Live The King!" am I the only one who thought of Megadeth?

    LOL, there's a good chance that Megadeth was playing in a YouTube tab while I was doing the final layout for this article.
  • 2 Hide
    adamovera , November 19, 2012 5:50 AM
    epilepticIs it Opera x64 or x86? I remember having tested Opera 12 and the startup was very slow. I'm still using 11.64 atm. The only thing keeping me from moving to Firefox is how sluggish the UI feels... I'd also have to find a new mail client.

    32-bit, they're all 32-bit.
  • 11 Hide
    randomizer , November 19, 2012 7:08 AM
    mayankleoboy11. IMHO, enabling these settings would have made Opera more competitive and this article fairer.


    Running every browser with its default configuration except one is not a fair test.
  • 2 Hide
    JOSHSKORN , November 19, 2012 7:09 AM
    adamovera32-bit, they're all 32-bit.

    Yes and this is unfortunate. We need more competition in the 64-bit browsing world. I currently use Waterfox. It's only other competition (that I'm aware of) is IE 64-bit.
  • 0 Hide
    stairmand , November 19, 2012 7:18 AM
    Would have been nice to include some fatures, the Pinch to Zoom on IE 10 makes that a winner for multi-touch ebnabled systems on its own.
  • 0 Hide
    assafbt , November 19, 2012 7:26 AM
    Ok, perhaps I missed something about the new composite scoring, but what I am always interested in is solely which browser has the least weak performances & behaviors, not really interested in who is good or best. With the new scoring we went back quite a bit, or am I missing something?
  • 4 Hide
    mayankleoboy1 , November 19, 2012 7:32 AM
    JOSHSKORNYes and this is unfortunate. We need more competition in the 64-bit browsing world. I currently use Waterfox. It's only other competition (that I'm aware of) is IE 64-bit.


    In most of these 'benchmarks', which basically run a few tight loops a gazillion times, 64 bit browsers will perform worse than 32 bit.
    Plus, Visual Studio 2010, on which all these browsers are compiled, does not many of the performance optimisations for 64 bit code.
  • 3 Hide
    Cryio , November 19, 2012 9:12 AM
    adamovera32-bit, they're all 32-bit.


    False, IE10 RTM on Windows 8 x64 is a 64 bit browser.
  • 2 Hide
    Cryio , November 19, 2012 9:13 AM
    mayankleoboy1In most of these 'benchmarks', which basically run a few tight loops a gazillion times, 64 bit browsers will perform worse than 32 bit. Plus, Visual Studio 2010, on which all these browsers are compiled, does not many of the performance optimisations for 64 bit code.


    Well, hopefully things change now with the release of Visual Studio 2012.
Display more comments