Web Browser Grand Prix 2: Running The Linux Circuit

Last week we showed Opera 10.60 to be the world's fastest Web browser. That was in the Windows world. But where do Chrome, Firefox, and Opera stand in Linux? Today we find out. Adding the Win 7 results, we'll also learn which OS has the speediest browser.

Alright folks. Today we have a special treat for you. The last two Web Browser Grand Prix stories received tons of reader-based feedback. But the comments in the first article resounded one question more loudly than the others: what about browser performance in Linux?

Ah, that's a good one.

Hot on the heels of Web Browser Grand Prix 2, we bring you The Linux Circuit. Using the exact same test system that we employed last time, we installed Ubuntu 10.04 LTS (Lucid Lynx) and the three major cross-platform Web browsers: Google Chrome, Mozilla Firefox, and Opera. Not only will be be declaring a winning Linux-based browser, but we'll also be comparing the results to those generated under Windows 7. Because the top three performers in Windows 7 happen to be Opera, Chrome, and Firefox, we can declare a winner that spans both platforms as well.

Dromaeo Drama

Over the past two articles we've had a little bit of controversy concerning Mozilla's Dromaeo JavaScript Benchmark. In the first Web Browser Grand Prix, Opera 10.50 scored far better than the other four Web browsers. Since it didn't affect Chrome's win either way, I decided to let Opera's highly dubious score persist. Chrome fans didn't seem to care, but Opera fans definitely let me know that the score is just swell, and I shouldn't be questioning it. In last week's sequel, Opera 10.60 again received an incredibly high score in the Dromaeo JavaScript test. That time, however, I disqualified Opera from the benchmark. Despite Opera 10.60 still handily taking the speed crown from Google Chrome, Opera fans really let me have it because of Chrome's slight edge in overall performance (including non-speed-related tests).

Now, just to set the record straight, the decision to remove Opera from the placing in the Dromaeo JavaScript benchmark did not affect the final outcome of the overall winner. If I had not disqualified Opera, making it the first-place finisher for that test, Chrome would still have been the overall performance champion. Though Chrome's win would have been by the absolute smallest possible margin, a quantifiable lead would have existed nonetheless. I don't like making these calls. I'd much prefer for everything to go smoothly. But stuff happens, and sometimes executive decisions need to be made.

Less than a week later, hindsight shows the decision to disqualify Opera as the right call. We have since contacted the three parties involved: Mozilla (author of the Dromaeo benchmarks), Opera, and Google. Below is an excerpt from a statement made by Mozilla's vice president of engineering, Mike Shaver:

Until recently, the Dromaeo benchmark, especially as regarded regular expression performance,  was very susceptible to optimizations specifically targeted at those weaknesses.  A JavaScript engine that was specifically altered to accommodate those tests could see astronomical score improvements by effectively subverting the work that is supposed to be done and timed.

Google brought this bug to our attention and requested that we fix the test. We agreed to, since it was clearly not testing anything useful. That test fix was completed last week.  (In the interim they added specific code to V8 to also "game" that aspect of the benchmark, for competitive reasons.

Well, there you have it. Whether or not Opera intentionally 'gamed' this benchmark is still up in the air. Without access to the source code, no one outside of Opera would know for sure. An Opera rep offered the initial idea that Profile-Guided Optimizations (PGO) may be to blame for the high Dromaeo scores seen in Windows. According to Opera, its OS X version (which doesn't yet have PGO enabled) does not achieve absurdly high scores in Dromaeo. The company had no results available for its Linux version, and the rep didn't know whether or not the penguin had PGO. Without any Linux data from Opera, we cannot further determine if PGO is truly the culprit. In the meantime, we are still waiting to hear back from the developers in Oslo for an official explanation for the high Dromaeo JavaScript scores.

As far as Google is concerned, thanks to Chromium, Mozilla has the smoking gun on the company's Dromaeo shenanigans. In the last article, I reported that Chrome 5.0.375.89 also yielded dramatically high Dromaeo JavaScript scores--much higher than the score that got Opera disqualified. Mozilla and Google both confirmed that the 5.0.375.70 and 5.0.375.86 scores are bunk. The final version of Chrome that was tested and published (5.0.375.99) scored 290.72. Google contends that a caching feature, added to 5.0.375.70 but then removed in 5.0.375.99, was to blame. The Mountain View company did nothing about the high Dromaeo scores back then, even though they were aware of it, because Opera hadn't either.

According to Mr. Shaver, the test fix was put into place shortly after our last piece was published. We have since re-run the Dromaeo JavaScript tests on both Opera 10.60 and Chrome 5.0.375.99 using our Windows 7 Ultimate 64-bit test system. Opera now gets a score of 211.5, while Chrome still earns around 290. Neither Mozilla nor Google have reason to doubt the Chrome 5.0.375.99 score. While the Opera disqualification from our previous piece stands, we'll update Opera's Windows 7 Dromeao JavaScript score to reflect the 'real' result in the charts for this article.

With this brouhaha behind us, and without further delay, let the benchmarking begin!

Create a new thread in the US Reviews comments forum about this subject
This thread is closed for comments
Comment from the forums
    Your comment
    Top Comments
  • adamovera
    weirdguy99Why not put firefox 4 into the equation?

    micr0beyes i would have loved to have seen the firefox 4 beta with the results. although great article.

    When it's final, I'll test it.
  • Other Comments
  • Tamz_msc
    The article that I was waiting for.How the tables have turned!
    Conclusion:Firefox is quite capable in both Linux and Windows.
    I'm using Firefox 4 beta and I find it pretty quick.
  • micr0be
    yes i would have loved to have seen the firefox 4 beta with the results. although great article.
  • opmopadop
    Interesting how FireFox chose to sacrifice speed for memory.
  • adamovera
    weirdguy99Why not put firefox 4 into the equation?

    micr0beyes i would have loved to have seen the firefox 4 beta with the results. although great article.

    When it's final, I'll test it.
  • Tamz_msc
    DamdamanI'll get berated for this I'm sure but will we see an OSX article on browsers as well?

    You are kidding,right?
  • arnweb
    Opera Turbo feature, is not mentioned here, it can boost speed in real surf. And also when we open a closed tab in Opera it opens them instantly, that's why Opera holds memory for closed, tab.
  • Sihastru
    Opera still can't render pages properly, still can't print content properly, and we waste our time with senseless tests of imperceptible speed.
  • The_King
    I dont think anyone using firefox will change to another browser even if it is Faster. I love my firefox :)
  • Tamz_msc
    Wow, Chrome uses more memory than Crysis on my computer. :lol:
  • Tamz_msc
    arnwebOpera Turbo feature, is not mentioned here, it can boost speed in real surf. And also when we open a closed tab in Opera it opens them instantly, that's why Opera holds memory for closed, tab.

    Opera Turbo increases page load times on slow connections.On my 2Mbps connection the time in which Opera Turbo connects to its servers is the time in which Google loads in Firefox.
  • lradunovic77
    I still don't see a point of this test especially as hardware gets better and better. Computer which runs on SSD or Raid SSD with 20Mb/s Internet Connection don't matter what browser you're on cause you can't see a damn difference.
  • LaloFG
    Then all browsers lie except Opera hehehe, don't blame me, or whatever, that don't change the result: The fast browser in the knowing universe.
  • haplo602
    I guess the tab recycle bin is responsible for high opera mem usage even after the tabs are closed. however I have not yet found a way to disable it so it gets to normal memory management. however I love the feature, to the mem issue is not that big a deal for me...

    also this is a nitpick, but please do name the article the ubuntu circuit. I do not think the results will be 100% identical on other linux distros.
  • whiz
    Gratuliere is probably what you meant...
  • Anonymous
    No. "Gratulerer" is norwegian, so is Opera ;-)
  • ta152h
    Does Linux even matter? Outside of hippies, does anyone use it?
  • tntom
    Looks like if you have a system with less ram FF is your choice browser. But overall it is stomped by Chrome.

    I too am using FF4 beta. It is noticeably faster with page load times than FF3. But I agree about only testing final releases.

    Thanks Adam Overa for including Linux in the Grand Prix.
  • WarraWarra
    haplo602 +1
    Correct depends on GCC version installed + lots of other factors and source compile or bin and who / what did the bin on what config.

    With Mozilla now using gcc 4.5.* the 3.6.6 firefox is slower than ie6 in xp.
    Upgraded to 3.5.11 and 3.0.18 on 2 xp boxes I have from 3.6.6 , about to do the same in my linux boxes.

    The gcc 4.5.* optimization issues is bitting hard on Mozilla.

    Firefox 4 beta has same gcc 4.5.* issues = slow.

    Someone at Mozilla will have to source build a complete gcc 4.5.* distro and build it about 3 times over on same code to make sure all is included / properly optimized + pray it is working correctly.

    Arch linux had GCC 4.5.* integrated already so should be a good place for mozilla to start from.

    Can not stand Opera appearance and layout too much vista / win7 feel. Used to love opera way back just after we lost netscape.

    Nice to see Opera coming back in win + lin.
  • WarraWarra
    ta152hDoes Linux even matter? Outside of hippies, does anyone use it?

    LOL good question.
    With Win vista annoying so many a lot of people jumped to linux dual boot or just linux. Win7 has caused another major wave of new linux users + recession made allot of people think about $500 for Microsoft office that messes up their old word documents ect and free openoffice / koffice etc. in linux.

    So free properly working office pc operating system + office apps so might as well go linux instead of $1300 to win7 replace 90% of your software and pray it works + then still loose half your data with win7 issues / bugs.

    And you can do this from a live cd or dvd no need to install makes it very attractive. Can also read / write to ntfs hfs+ and other partitions even with vista / win7 locks enabled, makes getting to your data and formating your win7 much easier when vista /win7 constantly fails to works / boot.

    Linux is good not great not excellent but for office use 100 times better than Vista /win7.
    Gaming in linux is still bad or about the same as Vista / win7 so have to dual boot and play game in XP.
  • adamovera
    I hear ya, I hear all of ya, promise. This article HAD to match WBGP2 in order to do the comparison - any cool, do-able changes suggested in the comments of WBGP2 and WBGP2:Linux will occur for WBGP3, if humanly possible. They're not useless, I'm sure 4GB is about the average amount for TH readers. Also, we did remove the memory testing and standards testing results from the final speed-only result, we left them in for the overall results. So there are two finals, if you don't like the memory testing, ignore the overall winner and go with the speed king.