Sign in with
Sign up | Sign in

Test Setup And Benchmark Suite

Chrome 27, Firefox 22, IE10, And Opera Next, Benchmarked
By

Our test hardware underwent some significant changes since Web Browser Grand Prix: Chrome 25, Firefox 19, And IE10. We're now on an Ivy Bridge-based Core i5 rig, upgraded from the equivalent Sandy Bridge parts. Our cable modem and ISP speeds are also better than last time. For now, the Web server and router remain the same, though both should be new by the next time we do one of these stories.

Our benchmark suite is also changing. First, that 40-tab workload in our memory tests is now timed using a stopwatch, and contributing to our overall wait time score (start-up and page load times).

Moving on to JavaScript, after we retired SunSpider from the test suite, WebKit updated the long-believed-abandoned JS performance test. We'll be running the newly-released SunSpider v1.0 to see if this benchmark's issues with Internet Explorer have been resolved.

KaizouMark makes a return now that IE9 is no longer in the mix, finally providing us with a long-overdue CSS3 performance test.

The final massive change to the benchmark line-up is in Hardware Acceleration. Both native HTML5 HWA and WebGL swapped out an old test for a newer one. First, we say goodbye to Psychedelic Browsing, and hello to CanvasMark 2013. Next, Airtight Interactive's WebGL Demo gets benched in favor of LUIC Cubes, a more intense metric that should be useful longer.

Test Setup And Benchmark Suite

Test System Specs


Operating System


Microsoft Windows 8 Enterprise (64-bit)

Processor
Intel Core i5-3570K @ 4.2 GHz (quad-core)
Motherboard
Gigabyte GA-Z77X-UD5H (rev 1.0, F14 BIOS)
Memory
16 GB Crucial DDR3 @ 1600 MT/s (4 x 4 GB)
Graphics
Gigabyte GeForce GTX 660 Ti 1 GB GDDR5 (PCIe 2.0 x16)
Storage
Seagate Barracuda 7200.12 500 GB SATA II 3Gb/s, 7200 RPM, 16 MB Cache
Optical
Asus DRW-24B1ST/BLK/B/AS
Power Supply
Corsair TX750W (750 W max)
Case
Zalman MS-800 Plus
CPU Cooler
NZXT Kraken X60 (closed-loop liquid cooler)
Monitor
AOC E2752Vh 27-inch LED (1920x1080)
Keyboard
Logitech Wireless Keyboard K320
Mouse
Logitech Wireless Trackball M570

Local Web Server Specs


Operating System
Ubuntu 12.04 LTS Server Edition "Precise Pangolin" (32-bit)
Processor
Intel Pentium 4 @ 2.41 GHz
Motherboard
Biostar P4M80-M4
Memory
768 MB DDR @ 333 MT/s
Storage
Western Digital Caviar SE WD1600AAJD, 160 GB EIDE, 7200 RPM
Extra Packages
Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH, Node.js, NPM

Network Specs


ISP Service
Cox Preferred (18 Mb/s down, 2 Mb/s up)
Modem
Cisco Telephony Modem DPQ3212 (DOCSIS 3)
Router
Linksys WRT54G2 V1

Benchmark Suite


Startup Time
Cold Start Time (Google SERP, Cached)
Hot Start Time (Google SERP, Cached)
Cold Start Time (Eight Tabs, Cached)
Hot Start Time (Eight Tabs, Cached)
Page Load Time
EEMBC BrowsingBench
40-Tab Load Time (Top 40 Websites)
JavaScriptRIABench JavaScript (Eight Tests)
Futuremark Peacekeeper v2.0
Rightware Browsermark v2.0
JSBench
WebKit SunSpider v1.0
DOMMozilla Dromaeo DOM (Core)
HTML5
Principled Technologies WebXPRT CP1
Impact HTML5 Benchmark
Hardware Acceleration
Facebook JSGameBench v0.4.1
HTML5 HWA
WebVizBench
CanvasMark 2013
WebGLLUIC Cubes
Scirra WebGL Performance Test
Memory Efficiency
Memory Usage (Single Tab)
Memory Usage (40 Tabs)
Memory Management (-39 Tabs)
Reliability
Proper Page Loads
SecurityBrowserscope Security
Standards ConformanceHTML5Test.com
The CSS3 Test
Ecmascript Language test262

While applicable links are included in the table above, we also have a public delicious account dedicated to Web Browser Grand Prix benchmark links. Detailed methodologies are explained on the individual benchmark pages.

We've highlighted Opera Next in the charts to emphasize that it's still in development and not part of the usual WBGP line-up.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 102 comments.
This thread is closed for comments
Top Comments
  • 13 Hide
    tomfreak , June 30, 2013 10:31 PM
    I dont know how useful in this review when they are tested all the browser on a 1155 super computer, nobody is going to tell the diff if the browser is 0.25sec faster. Get some Brazos, Atom and run the test, these are the platform have problems with web browsing.
Other Comments
  • -4 Hide
    Onus , June 30, 2013 9:07 PM
    No, the Onus is not on Google; I'm using www.startpage.com for my searches.

    While this is interesting, I still encounter built-in pages (such as on routers or other network devices) that will not render cleanly in Firefox, but are perfect in IE. More often than not though, pages that would be filled with nuisance ads and popups are cleaned up nicely by Firefox with AdBlock+ and NoScript.
  • 4 Hide
    soundping , June 30, 2013 9:22 PM
    A good test is rendering a heavy site like Huffington Post. They use a ton of flash and java scripts.
  • 4 Hide
    pharoahhalfdead , June 30, 2013 9:32 PM
    I would like to see benchmarks on page start up, and load times comparing ssd, hard drives, and ram drives. Maybe I missed these an a previous article, but I feel since ssd's and ram drives are getting more popular, benchmarks should prove or dispel the the 'so called' benefits they bring.

    I have both and start up times for IE are quick but page load times are horrendously slow, whereas FF has slow start up times but superfast page load times. It's possible that add-ons are contributing to that.
  • 0 Hide
    mikeynavy1976 , June 30, 2013 9:35 PM
    Ever since I have compared Firefox and Chrome I've always found Chrome to start much faster (I'm running the Dev channel and my wife uses the Stable channel and they both take maybe 1 - 2 seconds to start cold or hot). Only IE beats both of them. Maybe Firefox 22 is that much faster and worth a try, but seeing as Chrome did so well in most categories I'll probably stick with it.
  • 6 Hide
    EzioAs , June 30, 2013 10:00 PM
    Having move back to Firefox a couple years ago after Chrome, I don't intend to use Chrome (or any other browser) regularly anymore. I still give Chrome 2-3 tries a week (just to compare things) but Chrome isn't better than FF in 3 things:

    1) Pages load noticeably slower
    2) Memory usage is indeed high (as seen in the benchmark above)
    3) FF add-ons are much better than chrome extensions.

    I never noticed any startup time difference for both FF and Chrome; it's possible they're both fast enough that it doesn't even matter at this point. I also like the FF toolbars better although that's really more of a personal preference. I've never tried maxthon though; heard it's pretty good.
  • 0 Hide
    beoza , June 30, 2013 10:09 PM
    I've never really noticed a difference in browsers speed wise. Sure some load pages faster than others, and some have issues with certain pages. But in the end they all take me to the same place. I use Firefox 95% of the time at home with adblock+, if I encounter an issue I clear cache, if it still has issues I switch to IE 10, usually this is all I need to do but once in a while IE has problems with a page and I just move on to something else. At work I'm stuck using IE10. The speed of a browser can also be affected by other factors like the speed of your connection, how many devices are on your network, what tasks your doing on the computer at the same time like gaming, downloading, streaming movies (netflix, hulu), and your hardware. You're not going to get much responsiveness on a 5yr old celeron w/2gb ram and Win XP while you're downloading a game, watching youtube and the A/V suite scans your computer in the background, and there's 5 other people all sharing your 10Mbps network. Which describes probably 50% of the users out there in the real world.
  • 6 Hide
    Someone Somewhere , June 30, 2013 10:11 PM
    Opera Next (and every other browser) is a significant step down in terms of features/customization from the current version.

    I'll miss a hell of a lot of stuff when I move off Presto-based Opera.
  • -1 Hide
    ElDani , June 30, 2013 10:22 PM
    Well done on this test, I actually found the test results genuinely helpful and your summary/conclusion to be well thought out.

    Still, this test shows us once more, that no modern browser - I exclude Opera from this, since it isn't a maintained release anymore - must absolutely be replaced by the winner of such tests. If you don't mind performance weaknesses of the Internet Explorer in certain areas, or if your most-accessed websites don't require you to use a certain alternative, then even Microsoft's browser of choice can be okay for daily use (if only as an engine in products like Avant, Maxthon, etc).

    The one thing I'm a bit curious about: why does Opera Next suddenly behave so differently from Chrome? Yes, there's a difference between Chrome 27 (WebKit) and Opera Next (Blink = Chrome 28), but if that's the only reason for the browser's weaker showing, then the future of Chrome doesn't look too good. What's your take on this?
  • 13 Hide
    tomfreak , June 30, 2013 10:31 PM
    I dont know how useful in this review when they are tested all the browser on a 1155 super computer, nobody is going to tell the diff if the browser is 0.25sec faster. Get some Brazos, Atom and run the test, these are the platform have problems with web browsing.
  • 6 Hide
    ikefu , June 30, 2013 10:42 PM
    I've used FF for quite a while and have been very happy with the last few releases. Things definitely seems generally zippier and I've WAY less pages with load errors. I've also got my computer, tablet, and phone FireFox copies all sync'd together and love that it just works.
  • 3 Hide
    varun706 , June 30, 2013 10:57 PM
    When the hell did Intel release an i5 3770K ?
  • 3 Hide
    Someone Somewhere , June 30, 2013 11:05 PM
    Quote:
    We're now on an Ivy Bridge-based Core i5


    I'm guessing it should have said i5-3570K.
  • 6 Hide
    varun706 , June 30, 2013 11:08 PM
    Yep I'm gonna upgrade my Firefox.
  • 2 Hide
    Jak Atackka , June 30, 2013 11:35 PM
    Interesting test. I haven't heard of Opera Next, but now I'll definitely be keeping my eye on it. I use Google Chrome and have noticed it's slower loading speed (both to open Chrome and a webpage) the past update, and judging by the test results, the changes are significant.

    Can I make a request for your next test? Try comparing SSDs and HDDs in some of the tests, especially cold boot. They are becoming more and more popular, and at least with some of these tests, I imagine they do have a significant impact on performance.
  • 0 Hide
    Someone Somewhere , June 30, 2013 11:39 PM
    Opera Next is just the development/beta fork of Opera. They have the nice extra that you get separate data storage - you can have both installed at the same time.

    When the Chromium-based version becomes stable, it will be called Opera.
  • 1 Hide
    Someone Somewhere , June 30, 2013 11:55 PM
    Somewhat useless, yes. As they are from the same codebase they should perform relatively similar to on windows though.
  • 1 Hide
    LordConrad , June 30, 2013 11:57 PM
    Good on you Firefox, keep it up!

    I love customizability and plug-in support of Firefox. I also love the separate address and search bars. If I type "IBM" into the address bar I want to go directly to the website, if I want to search then I'll use the search bar. Unfortunately, whenever I install Firefox, I have to re-enable the "go directly to the website" ability in the address bar because, starting with Firefox 4, some dork at Mozilla changed the functionality of the address bar.
  • 2 Hide
    Someone Somewhere , July 1, 2013 12:03 AM
    You can do all that in Opera (though I'm still kind of lost as to why they keep the search bar, given they were the first to allow searching from the address bar).

    Up/down keys are your friend.

    One other pet peeve I have is that the autopredict in browsers have a habit of interpreting '192.168.1.1' as '192.168.1.104', if you visit the latter more frequently. Someone disable autopredict for IP addresses, and I will use your browser.
  • 3 Hide
    csf60 , July 1, 2013 1:03 AM
    I always wondered why you don't test flash performance. Is it because it only depends on adobe flash player and not the browser?
Display more comments