Sign in with
Sign up | Sign in

Performance Benchmarks: JavaScript

Web Browser Grand Prix 3: IE9 Enters The Race
By

For us, JavaScript testing is perhaps the most exciting part of the WBGP. Not so much because we're abnormally interested in how new Web browsers perform. We're actually interested in how the results of these tests compare to each other. They all test JavaScript, so they should all have more or less the same outcomes, right? We've found this is not always the case. Many reviews rely solely on one JavaScript benchmark; we don't. The reason for running redundant benchmarks is to make absolutely sure that the proper winner is chosen. In the past, we've seen different outcomes, re-arranged placing, and a total mismatch of scale in scoring different benchmarks that measure the same thing. 

Kraken

This time around we removed the Mozilla Dromaeo JavaScript benchmarks due to continued susceptibility to manipulation. Replacing Dromaeo JavaScript is, well, Mozilla's replacement for Dromaeo JavaScript: Kraken.

As you can see, the new Internet Explorer utterly destroys the competition in Kraken, finishing in less than a quarter of the time it takes second-place finisher Chrome. Opera, Safari, and Firefox round out the test, all three finishing between 10 000 and 15 000 milliseconds.

Update: 03/22/11 @ 3:00 PM PST

We received several emails and comments about the score achieved by Internet Explorer 9 in Kraken. Its score was dramatically higher on a second test system, so we went back over the data spreadsheet and chart spreadsheet. It appears that the actual figures did not survive intact from the data sheet to the chart sheet. Everyone who pointed to a misplaced decimal as the culprit is correct. IE9 scored 14092.6, not 1409.26 in Kraken. We also noticed the Safari score is supposed to be 14432.33, not 11432.33. This changes the placing to Chrome in first, Opera second, IE9 third, followed closely by Firefox 3.6.15 in fourth and Safari in fifth. The results tables have also been updated to reflect this change. This does not, however, change the final outcome. IE9 is still the clear winner, earning first place in 15 out of the 32 tests. That's significantly more first-place finishes than Chrome achieved to win WBGP1, or Opera in WBGP2. IE9 most definitely wins WBGP3. Sincere thanks to our eagle-eyed readers who spotted the chart discrepancy.

JSBenchmark

Chrome carries the JSBenchmark win by a significant margin, ahead of Opera. Safari and the new Internet Explorer take the third and fourth place spots, both scoring around 550 points. As seen in Kraken, Firefox 3.6.15 came in dead-last, but by a much greater margin in JSBenchmark. Overall, this is a very different outcome than the results from Kraken.

SunSpider

SunSpider is pretty important because it's essentially the go-to JavaScript test for most publications, as well as many manufacturers. Let's see how it compares to the other JavaScript tests.

IE9 comes in first, followed by Opera and Chrome in a near dead heat, both scoring 19 milliseconds. Safari takes fourth place, while Firefox brings up the rear. The results in SunSpider almost mirror those from Kraken.

Google V8 Benchmark

We decided to let the Google V8 benchmark remain for WBGP3. This benchmark has been the target of complaints from Mozilla, Opera, and scores of readers. When we ran Google V8 in Web Browser Grand Prix 2: Running The Linux Circuit, the results put Chrome ahead of Opera, despite every other JavaScript benchmark giving Opera the clear win. Since the main branch of WBGP is on Windows, and this aberration happened on Linux, we're giving V8 one more chance. If the results contradict other JavaScript tests in favor of Chrome, we'll simply disqualify it.

The placing here is Chrome, Opera, Safari, IE9, and Firefox--results that are identical to JSBenchmark. Although V8 again puts Chrome ahead of Opera, so do JSBenchmark and Kraken.

Interestingly, we have four different JavaScript benchmarks with two distinctly different outcomes. In fact, the only thing all four tests have in common is Firefox coming in last place. It appears the results of Kraken and SunSpider line up, as do the results of the V8 Benchmark and JSBenchmark. 

The only difference in overall placement from SunSpider to Kraken is the switch between Opera and Chrome for second place. Considering that second place is almost a tie in SunSpider, these two benchmarks support each other, even if the scale between the two is very different. Kraken puts IE9 in a solid lead, while SunSpider has the top three much closer together. Also, Firefox doesn't trail as far behind in Kraken.

JSBenchmark and Google V8 Benchmark both have identical placing, with Chrome in the lead. This result differs from Kraken and JSBenchmark which put IE9 in the lead, and that is the big issue here. IE9 handily won in both Kraken and SunSpider, but only placed fourth in V8 and JSBenchmark.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 244 comments.
This thread is closed for comments
Top Comments
  • 20 Hide
    andy5174 , March 21, 2011 4:25 AM
    Will there be a " Web Browser Grand Prix 4: Firefox 4 Enters The Race " after Firefox 4's release (22 Mar)?
  • 19 Hide
    turboflame , March 21, 2011 4:34 AM
    I like how this article was released one day before Firefox 4 is released.
Other Comments
  • 3 Hide
    pirateboy , March 21, 2011 4:17 AM
    I wonder how much microsoft paid tomshardware not to include Firefox 4 RC1
  • 1 Hide
    cruiseoveride , March 21, 2011 4:18 AM
    Now everyone is going to switch from Chrome to IE?
  • 0 Hide
    bison88 , March 21, 2011 4:18 AM
    Nice comparison Adam. I think people don't give MS credit for IE9 because of their history with web standards and lack of competition with updating their browser since they destroyed Netscape so many years ago. Hopefully they have their head in the game and continue with a somewhat constant update cycle. Currently the transition from IE8 to IE9 was somewhat FireFox'ish with speed instead of waiting for another revision of the OS. The browser definitely surprised me, some quirks here and there that are definitely noticeable and the Chrome/FF influence is obviously present. I wish them luck as the competition heats up.
  • 20 Hide
    andy5174 , March 21, 2011 4:25 AM
    Will there be a " Web Browser Grand Prix 4: Firefox 4 Enters The Race " after Firefox 4's release (22 Mar)?
  • 1 Hide
    illuminatuz , March 21, 2011 4:28 AM
    i felt the speed once i switched from firefox to chrome and now chrome to IE..
    but there is no mention of any addon/extentions for IE..
    i mean there is no comparision on who's got nice addons and startup time "with" addons.. since users tend to use web browsers with addons knowingly or un-knowingly..
    it would be better if you include those two comparisions as well.. and who takes advantage of GPU and all..
  • -2 Hide
    winner4455 , March 21, 2011 4:29 AM
    Won't take that long for IE to lag behind.
  • 1 Hide
    illuminatuz , March 21, 2011 4:33 AM
    winner4455Won't take that long for IE to lag behind.

    lol
    maybe not!!
    and FFS why doesnt that "submit my comment" button not working in IE!! I am using chrome for that ;(
  • 19 Hide
    turboflame , March 21, 2011 4:34 AM
    I like how this article was released one day before Firefox 4 is released.
  • 7 Hide
    dimamu15 , March 21, 2011 4:37 AM
    duck it, i stick to firefox.
  • 1 Hide
    Scanlia , March 21, 2011 4:38 AM
    Is there a wrong picture on the efficiency page? The 40tabs graph comes up as GUIMark 2?
  • 1 Hide
    IzzyCraft , March 21, 2011 4:38 AM
    The 40 Tabs chart is wrong for Efficiency Benchmarks: Memory Usage And Management it shows flash vector charting.

    pirateboyI wonder how much microsoft paid tomshardware not to include Firefox 4 RC1

    Then they would have to include every random nightly build of every browser? Comparing the latest release is fair you'll always get one browser or two that is near a new stable version but that's just how it works.

    winner4455Won't take that long for IE to lag behind.

    depends in Microsoft moves up an update schedule that doesn't just include making the browser more secure and fixing bugs. IE8 was pretty snappy back when it was first out it got old in like 3 months compared to other browsers though.
  • 0 Hide
    adamovera , March 21, 2011 4:43 AM
    pirateboyI wonder how much microsoft paid tomshardware not to include Firefox 4 RC1

    We only test final products in the WBGP.
    andy5174Will there be a " Web Browser Grand Prix 4: Firefox 4 Enters The Race " after Firefox 4's release (22 Mar)?

    turboflameI like how this article was released one day before Firefox 4 is released.

    Where are ya'll getting a set date from?
    illuminatuzi felt the speed once i switched from firefox to chrome and now chrome to IE..but there is no mention of any addon/extentions for IE.. i mean there is no comparision on who's got nice addons and startup time "with" addons.. since users tend to use web browsers with addons knowingly or un-knowingly..it would be better if you include those two comparisions as well.. and who takes advantage of GPU and all..

    We add/modify tests every time with the WBGP. We're already looking into those suggestions.
    ScanliaIs there a wrong picture on the efficiency page? The 40tabs graph comes up as GUIMark 2?

    Doh! Fixing that now.
  • 3 Hide
    andy5174 , March 21, 2011 4:51 AM
    Quote:
    Where are ya'll getting a set date from?


    http://groups.google.com/group/mozilla.dev.planning/browse_thread/thread/d7cd3c9e9650b1b3
  • -2 Hide
    molo9000 , March 21, 2011 4:53 AM
    Wait a second. Did they really include Safari? Who uses Safari for Windows?
    (I'm assuming they tested the Windows version of Safari, because why would you compare browsers for different systems?)

    I'm sticking with Opera on my Windows machine and Safari on my Mac.
  • 4 Hide
    Benihana , March 21, 2011 4:54 AM
    To be honest I did not read this article, because you're using the "latest and greatest" browsers, but somehow jumped a day early and didn't wait for FF4. Maybe you mentioned it in the article, but if not, then was it really so bad to wait an additional 1-2 days to do this?
  • 0 Hide
    DjEaZy , March 21, 2011 4:58 AM
    ... don't know... but this all is not my experience... why there is no firefox 4.0 RC2 ????
  • 0 Hide
    jezzarisky , March 21, 2011 5:01 AM
    adamoveraWhere are ya'll getting a set date from?



    http://www.theregister.co.uk/2011/03/16/mozilla_ship_firefox_4_22_march/

    Although most websites link to this website, so perhaps it was nothing more than a rumor.
  • 0 Hide
    andy5174 , March 21, 2011 5:11 AM
    Quote:
    Wait a second. Did they really include Safari? Who uses Safari for Windows?
    (I'm assuming they tested the Windows version of Safari, because why would you compare browsers for different systems?)

    I'm sticking with Opera on my Windows machine and Safari on my Mac.

    Opera sucks balls in memory management and lacks handy add-on such as Tab Mix Plus (which is same for Chrome). I would definitely move to Opera if it has these 2 issues fixed.
  • 0 Hide
    luc2k , March 21, 2011 5:12 AM
    https://wiki.mozilla.org/Firefox/4/Beta

    This article looks like a case of premature e.. something.
  • -3 Hide
    adamovera , March 21, 2011 5:12 AM
    andy5174http://groups.google.com/group/moz [...] 9e9650b1b3

    On the 16th he said RC1 would become the final code on the 22nd, but they released an RC2 since. I don't believe them anymore. Where is an official announcement or press release? RC1 was supposed to be here in November 2010. WBGP3 was going to feature FF4 at the end of 2010, then again at the end of this Feb. IE9 gave a date, so WBGP3 was made to coincide with its release. IF Firefox 4 launches tomorrow, it's purely coincidental. We stopped holding the WBGP series up for FF4 when MS officially gave their IE9 release date.
Display more comments