Skip to main content

Web Browser Grand Prix 2: Running The Linux Circuit

Introduction

Alright folks. Today we have a special treat for you. The last two Web Browser Grand Prix stories received tons of reader-based feedback. But the comments in the first article resounded one question more loudly than the others: what about browser performance in Linux?

Ah, that's a good one.

Hot on the heels of Web Browser Grand Prix 2, we bring you The Linux Circuit. Using the exact same test system that we employed last time, we installed Ubuntu 10.04 LTS (Lucid Lynx) and the three major cross-platform Web browsers: Google Chrome, Mozilla Firefox, and Opera. Not only will be be declaring a winning Linux-based browser, but we'll also be comparing the results to those generated under Windows 7. Because the top three performers in Windows 7 happen to be Opera, Chrome, and Firefox, we can declare a winner that spans both platforms as well.

Dromaeo Drama

Over the past two articles we've had a little bit of controversy concerning Mozilla's Dromaeo JavaScript Benchmark. In the first Web Browser Grand Prix, Opera 10.50 scored far better than the other four Web browsers. Since it didn't affect Chrome's win either way, I decided to let Opera's highly dubious score persist. Chrome fans didn't seem to care, but Opera fans definitely let me know that the score is just swell, and I shouldn't be questioning it. In last week's sequel, Opera 10.60 again received an incredibly high score in the Dromaeo JavaScript test. That time, however, I disqualified Opera from the benchmark. Despite Opera 10.60 still handily taking the speed crown from Google Chrome, Opera fans really let me have it because of Chrome's slight edge in overall performance (including non-speed-related tests).

Now, just to set the record straight, the decision to remove Opera from the placing in the Dromaeo JavaScript benchmark did not affect the final outcome of the overall winner. If I had not disqualified Opera, making it the first-place finisher for that test, Chrome would still have been the overall performance champion. Though Chrome's win would have been by the absolute smallest possible margin, a quantifiable lead would have existed nonetheless. I don't like making these calls. I'd much prefer for everything to go smoothly. But stuff happens, and sometimes executive decisions need to be made.

Less than a week later, hindsight shows the decision to disqualify Opera as the right call. We have since contacted the three parties involved: Mozilla (author of the Dromaeo benchmarks), Opera, and Google. Below is an excerpt from a statement made by Mozilla's vice president of engineering, Mike Shaver:

Until recently, the Dromaeo benchmark, especially as regarded regular expression performance, was very susceptible to optimizations specifically targeted at those weaknesses. A JavaScript engine that was specifically altered to accommodate those tests could see astronomical score improvements by effectively subverting the work that is supposed to be done and timed.Google brought this bug to our attention and requested that we fix the test. We agreed to, since it was clearly not testing anything useful. That test fix was completed last week. (In the interim they added specific code to V8 to also "game" that aspect of the benchmark, for competitive reasons.

Well, there you have it. Whether or not Opera intentionally 'gamed' this benchmark is still up in the air. Without access to the source code, no one outside of Opera would know for sure. An Opera rep offered the initial idea that Profile-Guided Optimizations (PGO) may be to blame for the high Dromaeo scores seen in Windows. According to Opera, its OS X version (which doesn't yet have PGO enabled) does not achieve absurdly high scores in Dromaeo. The company had no results available for its Linux version, and the rep didn't know whether or not the penguin had PGO. Without any Linux data from Opera, we cannot further determine if PGO is truly the culprit. In the meantime, we are still waiting to hear back from the developers in Oslo for an official explanation for the high Dromaeo JavaScript scores.

As far as Google is concerned, thanks to Chromium, Mozilla has the smoking gun on the company's Dromaeo shenanigans. In the last article, I reported that Chrome 5.0.375.89 also yielded dramatically high Dromaeo JavaScript scores--much higher than the score that got Opera disqualified. Mozilla and Google both confirmed that the 5.0.375.70 and 5.0.375.86 scores are bunk. The final version of Chrome that was tested and published (5.0.375.99) scored 290.72. Google contends that a caching feature, added to 5.0.375.70 but then removed in 5.0.375.99, was to blame. The Mountain View company did nothing about the high Dromaeo scores back then, even though they were aware of it, because Opera hadn't either.

According to Mr. Shaver, the test fix was put into place shortly after our last piece was published. We have since re-run the Dromaeo JavaScript tests on both Opera 10.60 and Chrome 5.0.375.99 using our Windows 7 Ultimate 64-bit test system. Opera now gets a score of 211.5, while Chrome still earns around 290. Neither Mozilla nor Google have reason to doubt the Chrome 5.0.375.99 score. While the Opera disqualification from our previous piece stands, we'll update Opera's Windows 7 Dromeao JavaScript score to reflect the 'real' result in the charts for this article.

With this brouhaha behind us, and without further delay, let the benchmarking begin!

  • Tamz_msc
    The article that I was waiting for.How the tables have turned!
    Conclusion:Firefox is quite capable in both Linux and Windows.
    I'm using Firefox 4 beta and I find it pretty quick.
    Reply
  • micr0be
    yes i would have loved to have seen the firefox 4 beta with the results. although great article.
    Reply
  • opmopadop
    Interesting how FireFox chose to sacrifice speed for memory.
    Reply
  • adamovera
    weirdguy99Why not put firefox 4 into the equation?micr0beyes i would have loved to have seen the firefox 4 beta with the results. although great article.When it's final, I'll test it.
    Reply
  • Tamz_msc
    DamdamanI'll get berated for this I'm sure but will we see an OSX article on browsers as well?You are kidding,right?
    Reply
  • arnweb
    Opera Turbo feature, is not mentioned here, it can boost speed in real surf. And also when we open a closed tab in Opera it opens them instantly, that's why Opera holds memory for closed, tab.
    Reply
  • Sihastru
    Opera still can't render pages properly, still can't print content properly, and we waste our time with senseless tests of imperceptible speed.
    Reply
  • The_King
    I dont think anyone using firefox will change to another browser even if it is Faster. I love my firefox :)
    Reply
  • Tamz_msc
    Wow, Chrome uses more memory than Crysis on my computer. :lol:
    Reply
  • Tamz_msc
    arnwebOpera Turbo feature, is not mentioned here, it can boost speed in real surf. And also when we open a closed tab in Opera it opens them instantly, that's why Opera holds memory for closed, tab.Opera Turbo increases page load times on slow connections.On my 2Mbps connection the time in which Opera Turbo connects to its servers is the time in which Google loads in Firefox.
    Reply