Web Browser Grand Prix 5: Opera 11.50, Firefox 5, And Chrome 12

Reliability Benchmarks: Proper Page Loads

The memory usage benchmarks have been part of the Grand Prix since WBGP2, and during the 40-tab test we get to observe several strange behaviors from the various Web browsers. Until now, these were merely observations, not something we specifically looked for. But the inability of Safari to load all 40 tabs at the same time and the need to reload several pages in Chrome fall into the category of reliability. Beginning with this competition, the WBGP now sports reliability benchmarks.

We jot down every page that needs a refresh due to broken or missing elements from the 40-tab memory usage test in order to get a sense of how reliably each browser loads Web pages under heavy load. Obviously, all of the contenders properly load a single page, so a more extreme scenario is really the only way these failures manifest themselves consistently. Examples of page load failures include: error pages, blank pages, missing/broken images, broken layouts, frozen/missing videos, and blank ads.

The chart below lists the average number of tabs that required a refresh due to broken or missing elements.

As you can see, Firefox deserves its reputation as the rock-steady browser. Mozilla's latest creation succeeds at loading every single element of every single Web page in all 40 tabs, all at once, every single time. This is something that we've noticed throughout the WBGP. But this time we recorded it. Opera ties for first, also requiring zero reloads during the three iterations of the 40-tab memory usage tests. Google Chrome is consistently one of the fastest Web browsers in the pack, yet it requires multiple reloads due to broken or unfinished pages under the exact same load. This time around, we recorded 5.333 reloads on average. While not the greatest for reliability, this is definitely an improvement over recent versions.

The chart shows that Microsoft Internet Explorer 9 required an average of 13.666 reloads, however, the story is actually much worse than what we could record. IE9 often required multiple reloads per tab, meaning that after all 40 tabs were loaded and we reloaded the broken ones, some of them remained broken. We consistently had trouble loading YouTube, Wikipedia, The Huffington Post, and C3 Metrics with Internet Explorer. There were many other broken sites during each iteration, but these four pages needed to be reloaded multiple times on each of the three iterations. IE9 also crashed after the initial load requiring the application to be restored on each iteration. Strangely enough, IE9 exhibits the bizarre behavior of loading simple white space-heavy Web sites like Wikipedia and Craigslist in all black.

While the chart shows that Safari only needs an average of one reload per iteration, its placement at the bottom of the chart is no mistake. Remember that Safari simply will not finish loading all 40 tabs at the same time, and that we have to open each site individually in a new tab. This fact alone puts Safari at the bottom. The single reload average is from loading sites individually.

So there you have it. For proper page loads, you just can't beat Mozilla Firefox or Opera. Speed demons Chrome and IE9 get the job done first, but the results are often sloppy; much more so in the case of IE9. Apple's Safari simply cannot compete on this field. For power users, the ability to properly load a ton of pages is what separates the men from the boys, and right now Firefox 5 and Opera 11.50 are Chuck Norris.

Create a new thread in the US Reviews comments forum about this subject
This thread is closed for comments
102 comments
    Your comment
    Top Comments
  • cadder
    Will you guys please investigate the SECURITY of each browser? I would use the one that is most secure even if it is slowest.
    28
  • adampower
    Wow, it seems like I upgrade my browsers every week.
    18
  • JOSHSKORN
    Now if just Google would release a 64-bit Chrome browser.
    18
  • Other Comments
  • adampower
    Wow, it seems like I upgrade my browsers every week.
    18
  • Anonymous
    somehow it seems that firefox is focussing more on benchmarks rather than actual real world usage.
    -9
  • Anonymous
    opera keeps impressing me throughout the WBGP
    9
  • JOSHSKORN
    Now if just Google would release a 64-bit Chrome browser.
    18
  • Tamz_msc
    First of all, this is the most thorough WBGP yet.
    Quote:
    somehow it seems that firefox is focussing more on benchmarks rather than actual real world usage.

    I don't think so - proper page loads and battery life are important considerations.

    I agree that Mozilla did not do a right thing in copying Chrome's release cycle, but at least they're trying - for example, they're trying hard in bringing down memory usage by increasing the garbage collection frequency(check this out in the Aurora and Nightly builds).

    WBGP is basically a test of speed, and Chrome may have won in that, but Firefox is not far behind. I can wait for two or three seconds for my page to load. You can easily bring down the page load times by using addons like AdBlock Plus.

    Even with the faster release cycle, this article clearly states that Firefox is still the most stable browser. Many people say that they've had numerous crashes, but its something wrong with their drivers or OS - I have not had a single crash since FF 4.0 beta 5 (or 7?), when they introduced hardware acceleration for the first time.

    Firefox remains the most customizable browser, while Opera has the most number of features out-of-the-box.

    So overall, according to me Firefox>=Opera>Chrome>IE 9> Safari.
    9
  • cadder
    Will you guys please investigate the SECURITY of each browser? I would use the one that is most secure even if it is slowest.
    28
  • Tamz_msc
    199194 said:
    Will you guys please investigate the SECURITY of each browser? I would use the one that is most secure even if it is slowest.

    That's easy: FF+AdBlock Plus+ NoScript+Ghostery+BrowserProtect
    4
  • ChiefTexas_82
    IE 9 is a speed demon? I droped IE because it started running like ****. I blame loading too many side programs after years on the web. So I wanted to try Chrome or Firefox. Being a Google fan already, I tried Chrome. So far it leaves my old IE8 in the dust. Except for a certain bug, I would say it has been an improvement in almost every way.
    -8
  • ChiefTexas_82
    I don't like how Nvidia's GPU auto-detect doesn't work on chrome.
    2
  • thartist
    Damn, Opera has it's flaws but it's nonetheless the one that does one thing best: browsing.
    -4
  • wheredahoodat
    I have personally along others have remained loyal to Firefox for years, but the latest moves have been are puzzling. Despite the benchmarks, they have not addressed bad cold start up problems (test on netbooks not on desktop rigs to find out), and the Firefox 4/5 new javascript engine has been a nightmare memory wise causing many to banish Firefox or remain with version 3.6 on older computers.

    Also instead of prioritizing the electrolysis project which would have made Firefox as snappy as Chrome, they are wasting their time on the Azure graphics project to replace a only few months old Cairo engine for marginal benefit because html 5 is still just the future, with no real significant penetration for the web.

    Mozilla can only push their base so far with their incompetence. Back then there was no real competition or alternative for a big market share, standard compliance browser. However there is now a real alternative with Chrome, and Firefox's gradual but definite market share hemorrhaging has shown that incompetence has consequences.
    9
  • beavermml
    which one is the best in term of security out of box?
    12
  • akorzan
    On the last page in the table and row "proper page loads," why is Opera in twice and no Safari?
    Surely, this is a typo.
    2
  • Maziar
    Thanks for the review,I was waiting for this !
    1
  • andy5174
    No doubt that Chrome is notably faster than Firefox so that I can feel the difference easily. However, Chrome lacks many useful features/add-ons offered by Firefox that I've been using for years. This is the main and only reason that Firefox is still my default browser.
    12
  • obarthelemy
    I'm still uncomfortable with those kind of tests. Especially because they don't take into account out-of-the-box features and creature comforts. I have all 4 installed (no Safari), and I find use Opera most, because it does very good Mouse Gestures, tab management, and synch. Without any addons ! I find Addons a very mixed blessing: I had plenty of headaches with FFox's, what with compatibility issues when upgrading FFox, bug and slow/ban/non-existent support... I find that having all required functionnality included in the base browser is a big plus, more-so for non-techies.
    1
  • Anonymous
    How can you place Firefox in Memory Usage as strong. This thing is leaking all over the place. Should you review your methodology?
    PS I m an early adopter of FF since 2.0 but I consider switching due to this memory issue.
    3
  • johnsmithhatesVLC
    Firefox is the only browser that can block all ads properly. It's also the most reliable at page loading as the WGP has proven. I really don't care about millisecond differences in page loads. Firefox is the best to me.
    14
  • lucas1024
    Thanks for including the reliability test! I had almost convinced myself to switch to Chrome from FF5, because of the memory usage, but not anymore - I do like my pages to load every time and I routinely have 30-40 tabs open.
    4
  • neiroatopelcc
    page 3 :
    64-bit Desktop Test System
    Operating System
    Microsoft Windows 7 Ultimate (32-bit)
    0