Web Browser Grand Prix: Chrome 25, Firefox 19, And IE10

Round-ups
By Adam Overa
published

Important versions of Chrome and Firefox were released since our last installment, and though we're ready to leave Windows 7 behind, IE10 brings us back for one more round. Naturally, our exhaustive suite of benchmarks undergoes a massive update as well.

Hardware Acceleration Performance

Native HTML5 Hardware Acceleration

This portion of the test suite remains unchanged, with WebVizBench and Psychedelic Browsing providing the native HWA results. The following chart is a geometric mean of these two benchmarks.

Internet Explorer takes the lead, leaving Firefox and Chrome duke it out for second place. Opera is totally out-gunned in native HWA.

The following two charts contain the individual results of WebVizBench and Psychedelic Browsing:

Image 1 of 2
WebVizBench

WebGL

Both of our WebGL tests get replaced by new benchmarks in this installment of the Grand Prix. Mozilla's WebGL spin of the FishIE benchmark was maxed out long ago by our modern test system, and we simply like the new duo better than WebGL Aquarium from Chrome Experiments.

Once again, only Google Chrome and Mozilla Firefox support WebGL by default.

Our first new test comes to us from the folks at Airtight Interactive. The aptly-named WebGL Demo renders a configurable number of multicolored, transparent, spinning cubes. We chose to render 2,000 cubes in order to bring our frame rates down into the 30 FPS territory.

This benchmark has Firefox in the lead at 37 FPS in Windows 7 and 34 FPS in Windows 8. Chrome takes second place at 30 frames per second on both platforms.

The WebGL Performance Test from Scirra is our other new WebGL-based benchmark. Unlike all our other tests in this category, it doesn't measure frames per second. Instead, the Scirra WebGL Performance Test continues to add objects until the frame rate drops below 30 FPS.

As with our previous set of WebGL performance tests, the new pair also conflicts. The Scirra test shows Chrome to be the more powerful browser, achieving nearly 133,000 objects on-screen before dipping under 30 FPS. Meanwhile, Firefox only manages about 60,000 objects before entering sub-prime performance territory.

The results of the two tests together are represented in the chart below.

Chrome's overwhelming win in the Scirra test place it about 25% ahead of Firefox in the WebGL composite score.

JSGameBench

Facebook's JSGameBench is an all-around hardware acceleration benchmark that covers both native HTML5 HWA and WebGL. Like the new Scirra test, this also works by adding more elements, while maintaining a steady playable frame rate.

This benchmark is hosted from our local Web server.

As usual, Firefox steals the show in the HWA test from Facebook. IE10 manages to take second place ahead of Google Chrome. IE9 places fourth, while Opera barely places at all. This test has Firefox and IE10 doing substantially better in Windows 8.

Hardware Acceleration Composite Score

The HWA composite is the geometric mean of JSGameBench, and the HTML5 and WebGL composite scores.

With all five HWA tests taken together, Firefox retains its lead, followed by Chrome in second place. Internet Explorer is in third, while Opera is hardly on the map at all, placing last.

This brings us to the end of the WBGP's performance section, also marking the final change in our test line-up. Next, we take a look at memory efficiency.

Adam Overa
84 Comments Comment from the forums
  • mayankleoboy1
    1. I am assuming that IE9 was included just to show improvements in IE10, and IE9 will be retired in the next WBGP

    2. " Opera jumps on-board the Chromium bandwagon" is false. Opera is using Webkit rendering engine for rendering. We dont know what the Javascript engine and JIT compiler is going to be.

    3. On page4, in "the average wait time" , you are taking the Geometric Mean. I think that using values less than 1 in a geometric mean is skewing the results. Can you take a Arithmetic Mean, and check please ?

    4. Memory usage reporting of IE9 and IE10 is completely bogus. For some workloads, in TaskManager, IE10 is seen using about 300MB memory, while it is actually using around 3GB RAM ! :O . My 4GB RAM is completely used up, and disk-paging starts occuring.

    5. I am completely mystified why quite a few browsers do better in JS benchmarks on Windows8 . I can understand getting better scores in WebGL or HW acceleration tests (because of potential better Dx) , but JavaScript execution does not use Dx. Any ideas ?
    Reply
  • alidan
    i wish we could get some older browsers in here to to see the difference newerones make.

    i personally am still on ff10, memory is really the reason i use fire fox, along with chrome. i would love to see how older versions stack up to newer ones.
    Reply
  • bison88
    Opera is definitely lagging, but a I love it to death. Hope the switch to the Webkit platform gives them the motivation to start leading some of the boards again as it's still a great browser. The memory problem can get absurd sometimes and builds up fast. I have to say IE 10 has shocked me just toying around with it on Windows 7. The thing is smooth and fast, something I haven't seen since the days when it dethroned Netscape.
    Reply
  • adamovera
    mayankleoboy11. I am assuming that IE9 was included just to show improvements in IE10, and IE9 will be retired in the next WBGP2. " Opera jumps on-board the Chromium bandwagon" is false. Opera is using Webkit rendering engine for rendering. We dont know what the Javascript engine and JIT compiler is going to be.3. On page4, in "the average wait time" , you are taking the Geometric Mean. I think that using values less than 1 in a geometric mean is skewing the results. Can you take a Arithmetic Mean, and check please ?4. Memory usage reporting of IE9 and IE10 is completely bogus. For some workloads, in TaskManager, IE10 is seen using about 300MB memory, while it is actually using around 3GB RAM ! . My 4GB RAM is completely used up, and disk-paging starts occuring.5. I am completely mystified why quite a few browsers do better in JS benchmarks on Windows8 . I can understand getting better scores in WebGL or HW acceleration tests (because of potential better Dx) , but JavaScript execution does not use Dx. Any ideas ?1. Yup
    2. The press release mentioned "Chromium", so I'm assuming WebKit/V8.
    3. Yes, you're right! The last timers went by milliseconds, so that wasn't an issue - the replacement charts should appear soon. Good catch!
    4. It seems pretty reasonable to me, basically mirrors Chrome in this regard.
    5. Nope
    Reply
  • caspy7
    @mayankleoboy1
    #2, incorrect, they have said they're going with Chromium's V8.

    @alidan
    Mozilla saw the err of their ways and got after memory. The most recent version of Firefox should beat version 10 in both memory and performance. The last benchmarks I saw had it beating all the other browsers in memory usage as well. (In this article you can see a snapshot of this in the "40 tabs" graphic. Like many techies, this is more what my browsing looks like.)
    Reply
  • mayankleoboy1
    adamovera1. Yup2. The press release mentioned "Chromium", so I'm assuming WebKit/V8.3. Yes, you're right! The last timers went by milliseconds, so that wasn't an issue - the replacement charts should appear soon. Good catch!4. It seems pretty reasonable to me, basically mirrors Chrome in this regard.5. Nope
    #2 : I must have forgotten the release. Thanks for the correction.
    #4 : I am not saying that IE10 uses excessive memory. I am saying that the total memory used by all the ieexplorer processes in the task manager is incorrect. For some image heavy pages, total memory usage of IE as reported by task manager is about 300MB, but total system memory usage gets around 3GB!. So if you close IE10, system memory goes from 90% full to 50% full.
    Reply
  • slomo4sho
    I currently use Firefox, Chrome, and IE 10. They each have their own advantages. Thanks for the write up. When can be expect the Android browser round up?
    Reply
  • xintox
    Really? I thought you were better than that at Toms.
    The composite hardware acceleration scores is most likely the main reason why IE9/10 is so far behind Firefox and Chrome on performane. Yet, from what I can tell, this composite score is heavily influenced by the WebGL scores, which is exclusive to Chrome and Firefox.
    In that respect, MS has at some point stated that they do not even wish to support WebGL, as it represents a significant security risk, as it gives the browser close access to the computer hardware.
    Long story short, your methods of calculating performance scores heavily favors Chrome and Firefox as they are the only ones to implement support for WebGL.

    Additionally, I wish you would make it more clear how you arrive at your composite scores and of course the final Performance Index. How do you add numbers that are so varied in nature, without some method of normalizing the numbers?
    Reply
  • srap
    @adamovera
    I heard that Futuremark Peacekeeper is unreliable, it used to miscalculate it's own benchmarks and it's still a black box so it may still be buggy. Do you know anything specific about it?
    Reply
  • abbadon_34
    Nice to see IE9 AND IE10, as well as Win 7 AND Win8. But I am curious how the final scores were calculated, seeing how WebGL was only supported on 2 browsers, as well as how the other incompatible tests were scored. A 0 vs not including can make a good deal of difference.
    Reply