Performance Benchmarks: JavaScript
For us, JavaScript testing is perhaps the most exciting part of the WBGP. Not so much because we're abnormally interested in how new Web browsers perform. We're actually interested in how the results of these tests compare to each other. They all test JavaScript, so they should all have more or less the same outcomes, right? We've found this is not always the case. Many reviews rely solely on one JavaScript benchmark; we don't. The reason for running redundant benchmarks is to make absolutely sure that the proper winner is chosen. In the past, we've seen different outcomes, re-arranged placing, and a total mismatch of scale in scoring different benchmarks that measure the same thing.
Kraken
This time around we removed the Mozilla Dromaeo JavaScript benchmarks due to continued susceptibility to manipulation. Replacing Dromaeo JavaScript is, well, Mozilla's replacement for Dromaeo JavaScript: Kraken.
As you can see, the new Internet Explorer utterly destroys the competition in Kraken, finishing in less than a quarter of the time it takes second-place finisher Chrome. Opera, Safari, and Firefox round out the test, all three finishing between 10 000 and 15 000 milliseconds.
Update: 03/22/11 @ 3:00 PM PST
We received several emails and comments about the score achieved by Internet Explorer 9 in Kraken. Its score was dramatically higher on a second test system, so we went back over the data spreadsheet and chart spreadsheet. It appears that the actual figures did not survive intact from the data sheet to the chart sheet. Everyone who pointed to a misplaced decimal as the culprit is correct. IE9 scored 14092.6, not 1409.26 in Kraken. We also noticed the Safari score is supposed to be 14432.33, not 11432.33. This changes the placing to Chrome in first, Opera second, IE9 third, followed closely by Firefox 3.6.15 in fourth and Safari in fifth. The results tables have also been updated to reflect this change. This does not, however, change the final outcome. IE9 is still the clear winner, earning first place in 15 out of the 32 tests. That's significantly more first-place finishes than Chrome achieved to win WBGP1, or Opera in WBGP2. IE9 most definitely wins WBGP3. Sincere thanks to our eagle-eyed readers who spotted the chart discrepancy.
JSBenchmark
Chrome carries the JSBenchmark win by a significant margin, ahead of Opera. Safari and the new Internet Explorer take the third and fourth place spots, both scoring around 550 points. As seen in Kraken, Firefox 3.6.15 came in dead-last, but by a much greater margin in JSBenchmark. Overall, this is a very different outcome than the results from Kraken.
SunSpider
SunSpider is pretty important because it's essentially the go-to JavaScript test for most publications, as well as many manufacturers. Let's see how it compares to the other JavaScript tests.
IE9 comes in first, followed by Opera and Chrome in a near dead heat, both scoring 19 milliseconds. Safari takes fourth place, while Firefox brings up the rear. The results in SunSpider almost mirror those from Kraken.
Google V8 Benchmark
We decided to let the Google V8 benchmark remain for WBGP3. This benchmark has been the target of complaints from Mozilla, Opera, and scores of readers. When we ran Google V8 in Web Browser Grand Prix 2: Running The Linux Circuit, the results put Chrome ahead of Opera, despite every other JavaScript benchmark giving Opera the clear win. Since the main branch of WBGP is on Windows, and this aberration happened on Linux, we're giving V8 one more chance. If the results contradict other JavaScript tests in favor of Chrome, we'll simply disqualify it.
The placing here is Chrome, Opera, Safari, IE9, and Firefox--results that are identical to JSBenchmark. Although V8 again puts Chrome ahead of Opera, so do JSBenchmark and Kraken.
Interestingly, we have four different JavaScript benchmarks with two distinctly different outcomes. In fact, the only thing all four tests have in common is Firefox coming in last place. It appears the results of Kraken and SunSpider line up, as do the results of the V8 Benchmark and JSBenchmark.
The only difference in overall placement from SunSpider to Kraken is the switch between Opera and Chrome for second place. Considering that second place is almost a tie in SunSpider, these two benchmarks support each other, even if the scale between the two is very different. Kraken puts IE9 in a solid lead, while SunSpider has the top three much closer together. Also, Firefox doesn't trail as far behind in Kraken.
JSBenchmark and Google V8 Benchmark both have identical placing, with Chrome in the lead. This result differs from Kraken and JSBenchmark which put IE9 in the lead, and that is the big issue here. IE9 handily won in both Kraken and SunSpider, but only placed fourth in V8 and JSBenchmark.
