Sign in with
Sign up | Sign in
Web Browser Grand Prix: Chrome 25, Firefox 19, And IE10
By ,
1. Possibly The Last "Top Four"

Here’s what has happened in the world of Web browsers since Which Web Browser Is Best Under Windows 8?:

Recent News And Events

11/13/12: The Release Preview of IE10 for Windows 7 is launched
11/18/12: Microsoft claims that WebKit is putting open Web standards in jeopardy
12/17/12: W3C Finalizes the Definition of HTML5
12/27/13: Mozilla intends to include H.264 support in Firefox 20
01/08/13: Mozilla releases Firefox 18 with new IonMonkey JavaScript engine
02/13/13: Opera announces that it’s switching to a Chromium base (WebKit and V8)
02/15/13: President of jQuery sees WebKit as another IE6 in the making
02/16/13: Opera Software purchases Skyfire Labs
02/22/13: Google releases Chrome 25
02/26/13: Microsoft releases IE10 for Windows 7
03/05/13: Chrome’s market share decreases slightly; Firefox, IE, and Safari benefit
03/06/13: EU imposes massive fine on Microsoft for missing Browser Ballot
03/07/13: Reports indicate that Google and Opera tipped the EU on Browser Ballot
03/07/13: Chrome, Firefox, and IE10 hacked at Pwn2Own, Opera and Safari not tested

Wow, what a couple of months, right? Firefox’s long-anticipated IonMonkey JavaScript engine finally landed, Microsoft and jQuery assail WebKit for its use of browser-specific extensions, Opera jumps on-board the Chromium bandwagon, and Google and Opera rat out Microsoft to the EU. Software people are always a riot.

Now, let’s quickly get acquainted with the today’s contenders before checking out the latest changes to our test suite.

2. Chrome, Firefox, Internet Explorer, Opera

As you may have heard, Safari for Windows is no longer with us. Thus, the top five are now the top four. Before Opera becomes yet another Webkit browser and we're down to the top three, let's quickly check the vitals of today's four contenders.

Windows 8 Web Browser Grand Prix Competitors

Chrome

Vendor:
Google
Debut:
9/02/2008
Layout Engine:
WebKit
JavaScript Engine:
V8
Current Version:
25
Supported Platforms:
Android, iOS, Linux, OS X, Windows
WBGP Championships:
Eight (Win7: 5 / Win8: 1 / Linux: 2)
Download Google Chrome

Firefox

Vendor:
Mozilla
Debut:
11/09/2004
Layout Engine:
Gecko 2.0
JavaScript Engine:
IonMonkey
Current Version:
19
Supported Platforms:
Android, Linux, OS X, Windows
WBGP Championships:
Three (Windows 7)
Download Mozilla Firefox

Internet Explorer

Vendor:
Microsoft
Debut:
8/16/1995
Layout Engine:
Trident
JavaScript Engine:
Chakra
Current Version:
10
Supported Platforms:
Windows 7, Windows 8, Windows RT
WBGP Championships:
Two (Windows 7)
Download Internet Explorer

Opera

Vendor:
Opera Software
Debut:
12/09/1996
Layout Engine:
Presto
JavaScript Engine:
Carakan
Current Version:
12.14
Supported Platforms:
Android, iOS, Linux, OS X, Windows
WBGP Championships:
Two (WinXP: 1 / Win7: 1)
Download Opera


Let's take a quick look at the test setup and current benchmark suite before we get started.

3. Test Setup And Benchmark Suite

While our test hardware has not changed since Which Web Browser Is Best Under Windows 8?, our benchmark suite underwent substantial changes. First, the startup times are now taken using a stopwatch, and the test pages are hosted from our local Web server. Using a stopwatch includes the time it takes for the actual applications (Web browsers) to open, as opposed to just timing how long it takes the browsers to load their tabs. By hosting the test pages from the local Web server, we cut out the instability and variation that comes with testing live pages.

As you may gave guessed, since we ditched our startup timers, we also ditched the page load timers that they're based on. We now use EEMBC's BrowsingBench to gauge page load times. This new test addresses several of the shortcomings that plagued our old page load timers; for instance, it tests both desktop and mobile pages, multiple pages on the same site, foreign sites, and it runs multiple iterations per test run.

Moving on to JavaScript, we finally retired SunSpider from the test suite. This benchmark hasn't been updated in several years, and there are far better JavaScript benchmarks available today (such as Kraken, which we're also retiring in favor of an even better test: Rightware Browsermark). Octane will not appear in the WBGP because the V8 portion still skews the results far in Chrome's favor. RIABench and Peacekeeper remain in the JS portion of the test suite. In order to minimize the effect of Dromaeo DOM in the final scoring, that test is now counted equally with the three JS tests in the final JavaScript/DOM composite score.

The HTML5 section also received several cuts, including GUIMark 2, Asteroids, and Smashcat. Only Impact remains. Joining it is the first consumer preview of Principled Technologies' WebXPRT, an HTML5 applications benchmark that frames the performance testing around common office Web apps, such as an image editor, stock tracker, and notes.

The final massive change to the benchmark lineup is in WebGL, both WebGL Solar System and Mozilla's WebGL FishIE Tank were replaced. Airtight Interactive's WebGL Demo and Scirra's WebGL Performance Test now make up that portion of the test suite.

Test Setup And Benchmark Suite

Test System Specs
Operating System 1
Microsoft Windows 7 Ultimate SP1 (64-bit)Microsoft Windows 7 Ultimate SP1 (64-bit)
Operating System 2
Microsoft Windows 8 (64-bit)Microsoft Windows 8 (64-bit)
Processor
Intel Core i5-2500K @ 3.3 GHz (quad-core)
Motherboard
Gigabyte GA-Z68XP-UD3 (F10 BIOS)
Memory
8 GB Crucial DDR3 @ 1333 MT/s (2 x 4 GB)
Graphics
Asus GeForce GTX 560 Ti 1 GB GDDR5 (PCIe 2.0 x16)
Storage
Seagate Barracuda 7200.12 500 GB SATA 3Gb/s, 7200 RPM, 16 MB Cache
Optical
Asus DRW-24B1ST/BLK/B/AS
Power Supply
Corsair TX750W (750 W max)
Case
Zalman MS-1000 HS2
CPU Cooler
Scythe Mugen 2 Revision B
Monitor
AOC E2752Vh 27-inch LED (1920x1080)
Keyboard
Logitech Wireless Keyboard K320
Mouse
Logitech Wireless Trackball M570
Local Web Server Specs
Operating System
Ubuntu 12.04.1 LTS Server Edition "Precise Pangolin" (32-bit)
Processor
Intel Pentium 4 @ 2.41 GHz
Motherboard
Biostar P4M80-M4
Memory
768 MB DDR @ 333 MT/s
Storage
Western Digital Caviar SE WD1600AAJD, 160 GB EIDE, 7200 RPM
Extra Packages
Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH, Node.js, NPM
Network Specs
ISP Service
Cox Preferred (18 Mb/s down, 2 Mb/s up)
Modem
Arris Touchstone Telephony Modem TM502G
Router
Linksys WRT54G2 V1
Benchmark Suite
Startup Time
Cold Start Time (Google SERP, Cached)
Hot Start Time (Google SERP, Cached)
Cold Start Time (Eight Tabs, Cached)
Hot Start Time (Eight Tabs, Cached)
Page Load Time
EEMBC BrowsingBench
JavaScriptRIABench JavaScript (Eight Tests)
Futuremark Peacekeeper v2.0
Rightware Browsermark v2.0
DOMMozilla Dromaeo DOM (Core)
HTML5
Principled Technologies WebXPRT CP1
Impact HTML5 Benchmark
Hardware Acceleration
Facebook JSGameBench v0.4.1
HTML HWA
WebVizBench
Psychedelic Browsing
WebGLAirtight Interactive WebGL Demo
Scirra WebGL Performance Test
Memory Efficiency
Memory Usage (Single Tab)
Memory Usage (40 Tabs)
Memory Management (-39 Tabs)
Memory Management (-39 Tabs, Two Additional Minutes)
Reliability
Proper Page Loads
SecurityBrowserscope Security
Standards ConformanceHTML5Test.com
The CSS3 Test
Ecmascript Language test262


While applicable links are included in the table above, we also have a public delicious account dedicated to Web Browser Grand Prix benchmark links.

Detailed methodologies are explained on the individual benchmark pages.

4. Startup Time

As we explained, these startup times have changed from a composite page load timer to a simple stopwatch. As a result, they'll be higher than any previous article's because we are timing from the point the application is launched to the point that all tabs report fully-loaded content. Before, we timed from the point the application window appeared to the time that all tabs finished loading. While a stopwatch isn't as precise as timer script output, it's nonetheless more reflective of real-world wait times.

As usual, we are timing startups both cold (first opened upon boot) and hot (reopened), in both single- and eight-tab variations. Let's begin with the single-tab cold start.

Single Tab, Cold

This measure is reflective of the time it takes to open a browser and load a single page immediately after turning on your computer.

Ironically, it's IE9 that demonstrates the fastest single-tab cold start, possibly the most common startup scenario, at just under three and a half seconds. Firefox places a close second, followed by Chrome at nearly double IE9's time. Oddly enough, IE10 for Windows 7 places fourth.

Under Windows 8, however, it's another story, with IE10 beating the rest at just 3.7 seconds. Opera places dead last on both versions of Windows, though it seems to do substantially better under Windows 8. 

Single Tab, Hot

Whenever you re-open a previously-closed browser in the same session, you're looking at hot startup time. This result pertains to users who prefer a single homepage.

IE9 and 10 basically tie for first place in single-tab hot starts. In Windows 7, Chrome shares the spotlight with Microsoft's browser, though it takes last place in Windows 8. Firefox and Opera essentially achieve the same times on either OS.

OK, single-tab times are great for Mom and Dad, but what about us power users who have our browsers set to open multiple pages, or to open tabs from the last session, or simply always have several pinned tabs at any given time? That's where the eight-tab tests come in. Why eight tabs? Simple, Internet Explorer only allows for up to eight tabs in a home tab group.

Eight Tabs, Cold

Again, the cold start times are indicative of when you first open your browser upon booting your PC.

Here, IE9 and Firefox share the top spot in Windows 7 at around 11.5 seconds. The eternal rivals are followed by Opera and Chrome, both at around 12.6 seconds, with IE10 bringing up the rear at over 14 seconds. Windows 8 is a completely different story, with Opera taking first place at less than nine seconds, followed by Firefox and Chrome. IE10 takes nearly 20 seconds!

Eight Tabs, Hot

When the browser is simply being re-opened, Internet Explorer once again assumes the lead, with IE10 outperforming IE9 by a full second on Windows 7. Chrome matches IE9 for second place on Windows 7, but falls to third on Windows 8. Opera and Firefox round out the placing on either OS in the 4-5 second range.

Average Wait Time

This figure is the average of all four startup scenarios, and it should give a general sense of how long each browser makes you wait in relation to the others.

Chrome should provide the lowest wait times (up to eight tabs) on Windows 7 with an average of just 4.5 seconds. Meanwhile, Firefox is the victor on Windows 8 and the second-place finisher on Windows 7. IE9 places a close third on Windows 7, followed by IE10. Opera places last on Windows 7, but earns second place on Windows 8. Chrome falls to a distant third place in Windows 8, while the platform's native IE10 takes the last place position.

5. Page Load Time

Again, we replaced our page load timers with the industry-standard BrowsingBench from The Embedded Microprocessor Benchmark Consortium (EEMBC). Unlike our page load timers, which used eight of the top 40 websites in the U.S., this benchmark includes foreign sites (a top reader request). Mobile pages and multiple pages on the same site fulfill two more wishes from our readers, and multiple iterations per test run means more even runs.

The finishing order in this test remains constant on both platforms: Chrome, Opera, IE10 (IE9), Firefox. While all browsers do slightly better in Windows 8, the boost that Firefox receives is quite substantial.

Now let's see if Mozilla's new IonMonkey JavaScript engine kicks Firefox into overdrive.

6. JavaScript And DOM Performance

JavaScript

We say a long overdue goodbye to SunSpider in this installment. Kraken also gets the boot, though it is being replaced by Rightware's Browsermark 2.0 (the first version was mobile-only). We're also vetoing the inclusion of Google's Octane for the very same reason that V8 is no longer in the Grand Prix: Chrome's margin of victory is contrary, and potentially crippling to the other JS results. RIABench and Futuremark's Peacekeeper remain, bringing our total number of JS tests to just three.

This benchmark has the browsers finishing in exactly the same order on both versions of Windows: Chrome, Opera, Firefox, IE10, IE9.

The charts below contain the results of the eight individual RIABench JavaScript tests.

Windows 7Windows 7Windows 8Windows 8

Peacekeeper places Chrome in the lead on both platforms, followed by Opera. Firefox takes third in the newer version of Windows, while the Internet Explorers beat Mozilla's browser in Windows 7. Once again, both Opera and Firefox exhibit better scores in Windows 8.

Our newest JavaScript performance benchmark shows Chrome to be in a commanding lead, followed by Opera on Windows 7 and Firefox in Windows 8; the browsers reverse positions in third place. IE9 actually beats IE10 in this test, though not by very much. Firefox yet again favors Windows 8.

DOM

The DOM Core tests in Mozilla's Dromaeo benchmark still serve as our sole DOM test, although now it is given even weight to the three JavaScript tests in our JS/DOM composite score.

The placing order is the same for both OSes: Chrome, followed closely by Firefox, and Opera in a distant third place. The IE duo falls to an even more distant fourth/fifth. Firefox again favors the newer Windows.

JavaScript/DOM Composite Score

This result is the geometric mean of the three JS tests and Dromaeo.

Chrome is the obvious winner in this category, with Firefox placing second, followed closely by Opera. IE10 takes fourth place, with its own predecessor placing last. It seems that Firefox's new IonMonkey JavaScript engine is unable to unseat Chrome from the JS throne.

Alright, now it's time to check out performance results for next-gen technologies like HTML5 and CSS3, followed by hardware acceleration.

7. HTML5 Performance

As with our JavaScript line-up, the HTML5 tests also underwent a facelift. We're no longer using GUIMark 2, Asteroids, or the Smashcat canvas test due to their age. Mandelbrot Set In HTML5 had to sit this one out due to the return of IE9. That leaves us with just Impact and a new test by Principled Technologies called WebXPRT.

Impact

This benchmark is a simulation of the Impact HTML5 game engine in action, and should be pretty indicative of simple HTML5 platformers.

The order that the browsers place in this test is identical on both platforms: IE10, Firefox, Chrome, Opera, and IE9. While the placing order remains the same across OSes, the scores are definitely not the same. IE10, Firefox, and Chrome all enjoy substantially better scores in Windows 8, and even Opera enjoys a moderately higher score.

WebXPRT

WebXPRT is a cross-platform HTML5 benchmark framed in the scenario of productivity Web apps. This test includes photo effects and face detection to represent tasks of an image editor, a stocks dashboard with tables and charts, and an offline note app. We used the Consumer Preview 1 build of this benchmark since it still isn't final. While we typically wouldn't do that, unlike almost every other pre-release benchmark we try, this one presented no problems on any of our browsers or platforms.

WebXPRT shows that Forefox is the browser best suited towards office-oriented HTML5-based Web apps, with Chrome taking a very close second place on both platforms. IE10 trails, but still secures a respectable third-place finish, followed by Opera in fourth. IE9 does not compare well to the newer contenders in this benchmark. All of the browsers again perform better under Windows 8.

HTML5 Composite Score

This score is a geometric mean of the Impact and WebXPRT results.

Firefox is the clear winner, though Chrome is not very far behind. IE10 also shows favorably, while Opera is noticeably behind the pack in this particular metric. IE9 is simply out-classed.

Unfortunately, we have no CSS3 tests, since we had to bring back IE9. Fortunately, we can't imagine a reason to test that browser again, so look forward to seeing Kaizoumark alongside our HTML5 benchmarks in the future.

Hear that GPU cooler spinning up? Sounds like it's time for hardware acceleration performance testing in HTML5 and WebGL.

8. Hardware Acceleration Performance

Native HTML5 Hardware Acceleration

This portion of the test suite remains unchanged, with WebVizBench and Psychedelic Browsing providing the native HWA results. The following chart is a geometric mean of these two benchmarks.

Internet Explorer takes the lead, leaving Firefox and Chrome duke it out for second place. Opera is totally out-gunned in native HWA.

The following two charts contain the individual results of WebVizBench and Psychedelic Browsing:

WebVizBenchWebVizBenchPsychedelic BrowsingPsychedelic Browsing

WebGL

Both of our WebGL tests get replaced by new benchmarks in this installment of the Grand Prix. Mozilla's WebGL spin of the FishIE benchmark was maxed out long ago by our modern test system, and we simply like the new duo better than WebGL Aquarium from Chrome Experiments.

Once again, only Google Chrome and Mozilla Firefox support WebGL by default.

Our first new test comes to us from the folks at Airtight Interactive. The aptly-named WebGL Demo renders a configurable number of multicolored, transparent, spinning cubes. We chose to render 2,000 cubes in order to bring our frame rates down into the 30 FPS territory.

This benchmark has Firefox in the lead at 37 FPS in Windows 7 and 34 FPS in Windows 8. Chrome takes second place at 30 frames per second on both platforms.

The WebGL Performance Test from Scirra is our other new WebGL-based benchmark. Unlike all our other tests in this category, it doesn't measure frames per second. Instead, the Scirra WebGL Performance Test continues to add objects until the frame rate drops below 30 FPS.

As with our previous set of WebGL performance tests, the new pair also conflicts. The Scirra test shows Chrome to be the more powerful browser, achieving nearly 133,000 objects on-screen before dipping under 30 FPS. Meanwhile, Firefox only manages about 60,000 objects before entering sub-prime performance territory.

The results of the two tests together are represented in the chart below.

Chrome's overwhelming win in the Scirra test place it about 25% ahead of Firefox in the WebGL composite score.

JSGameBench

Facebook's JSGameBench is an all-around hardware acceleration benchmark that covers both native HTML5 HWA and WebGL. Like the new Scirra test, this also works by adding more elements, while maintaining a steady playable frame rate.

This benchmark is hosted from our local Web server.

As usual, Firefox steals the show in the HWA test from Facebook. IE10 manages to take second place ahead of Google Chrome. IE9 places fourth, while Opera barely places at all. This test has Firefox and IE10 doing substantially better in Windows 8.

Hardware Acceleration Composite Score

The HWA composite is the geometric mean of JSGameBench, and the HTML5 and WebGL composite scores.

With all five HWA tests taken together, Firefox retains its lead, followed by Chrome in second place. Internet Explorer is in third, while Opera is hardly on the map at all, placing last.

This brings us to the end of the WBGP's performance section, also marking the final change in our test line-up. Next, we take a look at memory efficiency.

9. Memory Efficiency

The memory efficiency tests are not changed. In this metric, we open a Web browser with one tab, record memory usage, open 39 additional tabs, and re-record the memory usage total. We then close the additional 39 tabs and record again. We wait one minute and record the usage one more time. The first figure is subtracted from the last in order to see how much "bloat" remains after decreasing the workload (closing tabs).

Chrome takes the lead in Windows 7, with just 76 MB more after closing all 39 additional tabs. IE9 takes second place at 113 MB over the single-tab total. IE10 is close behind in third place at just 121 MB, while Firefox places fourth. Opera still clings onto nearly half of a gigabyte of data from the 39 closed tabs, putting it in last place. In Windows 8, IE10 and Chrome essentially share the lead, followed by Firefox and Opera.

The charts below contain the four individual memory readings.

Single TabSingle Tab40 Tabs40 Tabs

-39 Tabs-39 TabsOne Additional MinuteOne Additional Minute

The next test is actually performed during the memory efficiency testing cycle...

10. Page Load Reliability And Security

Page Load Reliability

When we opened the additional 39 tabs in our memory test, we made sure that each one was fully loaded, and then recorded the number of tabs that required a reload due to missing elements or broken formatting. These observations represent how reliable each browser is at completely and properly rendering pages as they're being subjected to a heavy workload.

Opera once again takes the lead for properly loading pages, averaging one reload in Windows 7 and three in Windows 8. Chrome places second on both versions of Windows, though it shares that spot with Firefox in Windows 8. Back to Windows 7, though. IE9 finishes in third place, followed by Firefox and IE10.

Interestingly, IE10 has more page-loading issues than IE9.

Security

BrowserScope's simple 17-point security checklist remains our sole measure of browser security.

Chrome takes the lead with just one failed checkpoint. IE10 steals second place, passing 14 of the 17 tests. Firefox and IE9 tie for third place, while Opera winds up in last place again.

Next up is standards conformance, followed by the winner's circle.

11. Standards Conformance

The chart below contains our standards conformance composite score, which is an average percentile "grade" of the three standards conformance benchmark scores. The three benchmarks that make up our conformance grade are: HTML5Test.com, The CSS3 Test, and Ecmascript Language test262. So, the grade is made up of equal parts HTML5, CSS3, and JavaScript.

Chrome takes the lead in standards conformance with a grade of 84%. Firefox places second at 77%, trailed by Opera just one percentage point behind. IE10 rates 70%, and although that's not exactly good, it's substantially better than IE9 at just 50%.

The charts below contain the individual benchmark results of our three standards conformance benchmarks.

HTML5HTML5CSS3CSS3JavaScriptJavaScript

12. Windows WBGP Winner's Circle

Before we crown a champion, let's see where the contenders stand in pure performance. The chart below only factors in the performance-based testing (startup time, page load time, JavaScript/DOM, HTML5, and HWA).

Wow. When we look strictly at the performance-oriented metrics, we almost have different winners for each version of Windows. While Chrome clearly takes the lead on Windows 7, Firefox almost scores a victory in Windows 8. IonMonkey narrowed the performance gap, just not quite enough. IE10 takes a distant third place under either OS, with IE9 in tow. Opera pitifully places last in performance testing, achieving one-third of the Chrome and Firefox scores.

Now we'll add efficiency, reliability, security, and standards conformance to the mix and see how the standings change. 

While it was a close race, Chrome's showing in all four of the remaining non-performance metrics was superior to Firefox, closing the deal. Firefox does place a very close second, especially in Windows 8, where it has an apparent advantage. In fact, looking at the performance index reveals that Firefox receives an even larger boost from the new version of Windows than IE10.

Speaking of, Microsoft's newest browser doesn't exactly shake the earth on either version of the company's ubiquitous OS, earning a mediocre third place. Opera and IE9 share last place, halving Chrome's score. It's hard to believe that the Norwegian browser went toe-to-toe with Google just a few short years ago. With version 12 a continuing disappointment, perhaps it's best that Opera join the ranks of Chrome clones.

Google Chrome remains the reigning Web Browser Grand Prix Champion for Windows (and Linux). Congratulations on the ninth WBGP victory!