Sign in with
Sign up | Sign in

Reliability Benchmarks: Proper Page Loads

Web Browser Grand Prix 5: Opera 11.50, Firefox 5, And Chrome 12
By

The memory usage benchmarks have been part of the Grand Prix since WBGP2, and during the 40-tab test we get to observe several strange behaviors from the various Web browsers. Until now, these were merely observations, not something we specifically looked for. But the inability of Safari to load all 40 tabs at the same time and the need to reload several pages in Chrome fall into the category of reliability. Beginning with this competition, the WBGP now sports reliability benchmarks.

We jot down every page that needs a refresh due to broken or missing elements from the 40-tab memory usage test in order to get a sense of how reliably each browser loads Web pages under heavy load. Obviously, all of the contenders properly load a single page, so a more extreme scenario is really the only way these failures manifest themselves consistently. Examples of page load failures include: error pages, blank pages, missing/broken images, broken layouts, frozen/missing videos, and blank ads.

The chart below lists the average number of tabs that required a refresh due to broken or missing elements.

As you can see, Firefox deserves its reputation as the rock-steady browser. Mozilla's latest creation succeeds at loading every single element of every single Web page in all 40 tabs, all at once, every single time. This is something that we've noticed throughout the WBGP. But this time we recorded it. Opera ties for first, also requiring zero reloads during the three iterations of the 40-tab memory usage tests. Google Chrome is consistently one of the fastest Web browsers in the pack, yet it requires multiple reloads due to broken or unfinished pages under the exact same load. This time around, we recorded 5.333 reloads on average. While not the greatest for reliability, this is definitely an improvement over recent versions.

The chart shows that Microsoft Internet Explorer 9 required an average of 13.666 reloads, however, the story is actually much worse than what we could record. IE9 often required multiple reloads per tab, meaning that after all 40 tabs were loaded and we reloaded the broken ones, some of them remained broken. We consistently had trouble loading YouTube, Wikipedia, The Huffington Post, and C3 Metrics with Internet Explorer. There were many other broken sites during each iteration, but these four pages needed to be reloaded multiple times on each of the three iterations. IE9 also crashed after the initial load requiring the application to be restored on each iteration. Strangely enough, IE9 exhibits the bizarre behavior of loading simple white space-heavy Web sites like Wikipedia and Craigslist in all black.

While the chart shows that Safari only needs an average of one reload per iteration, its placement at the bottom of the chart is no mistake. Remember that Safari simply will not finish loading all 40 tabs at the same time, and that we have to open each site individually in a new tab. This fact alone puts Safari at the bottom. The single reload average is from loading sites individually.

So there you have it. For proper page loads, you just can't beat Mozilla Firefox or Opera. Speed demons Chrome and IE9 get the job done first, but the results are often sloppy; much more so in the case of IE9. Apple's Safari simply cannot compete on this field. For power users, the ability to properly load a ton of pages is what separates the men from the boys, and right now Firefox 5 and Opera 11.50 are Chuck Norris.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 102 comments.
This thread is closed for comments
Top Comments
  • 28 Hide
    cadder , July 7, 2011 5:05 AM
    Will you guys please investigate the SECURITY of each browser? I would use the one that is most secure even if it is slowest.
  • 18 Hide
    JOSHSKORN , July 7, 2011 4:54 AM
    Now if just Google would release a 64-bit Chrome browser.
  • 18 Hide
    adampower , July 7, 2011 4:17 AM
    Wow, it seems like I upgrade my browsers every week.
Other Comments
  • 18 Hide
    adampower , July 7, 2011 4:17 AM
    Wow, it seems like I upgrade my browsers every week.
  • -9 Hide
    Anonymous , July 7, 2011 4:23 AM
    somehow it seems that firefox is focussing more on benchmarks rather than actual real world usage.
  • 9 Hide
    Anonymous , July 7, 2011 4:34 AM
    opera keeps impressing me throughout the WBGP
  • 18 Hide
    JOSHSKORN , July 7, 2011 4:54 AM
    Now if just Google would release a 64-bit Chrome browser.
  • 9 Hide
    Tamz_msc , July 7, 2011 4:58 AM
    First of all, this is the most thorough WBGP yet.
    Quote:
    somehow it seems that firefox is focussing more on benchmarks rather than actual real world usage.

    I don't think so - proper page loads and battery life are important considerations.

    I agree that Mozilla did not do a right thing in copying Chrome's release cycle, but at least they're trying - for example, they're trying hard in bringing down memory usage by increasing the garbage collection frequency(check this out in the Aurora and Nightly builds).

    WBGP is basically a test of speed, and Chrome may have won in that, but Firefox is not far behind. I can wait for two or three seconds for my page to load. You can easily bring down the page load times by using addons like AdBlock Plus.

    Even with the faster release cycle, this article clearly states that Firefox is still the most stable browser. Many people say that they've had numerous crashes, but its something wrong with their drivers or OS - I have not had a single crash since FF 4.0 beta 5 (or 7?), when they introduced hardware acceleration for the first time.

    Firefox remains the most customizable browser, while Opera has the most number of features out-of-the-box.

    So overall, according to me Firefox>=Opera>Chrome>IE 9> Safari.
  • 28 Hide
    cadder , July 7, 2011 5:05 AM
    Will you guys please investigate the SECURITY of each browser? I would use the one that is most secure even if it is slowest.
  • 4 Hide
    Tamz_msc , July 7, 2011 5:10 AM
    Quote:
    Will you guys please investigate the SECURITY of each browser? I would use the one that is most secure even if it is slowest.

    That's easy: FF+AdBlock Plus+ NoScript+Ghostery+BrowserProtect
  • -8 Hide
    ChiefTexas_82 , July 7, 2011 5:43 AM
    IE 9 is a speed demon? I droped IE because it started running like ****. I blame loading too many side programs after years on the web. So I wanted to try Chrome or Firefox. Being a Google fan already, I tried Chrome. So far it leaves my old IE8 in the dust. Except for a certain bug, I would say it has been an improvement in almost every way.
  • 2 Hide
    ChiefTexas_82 , July 7, 2011 5:45 AM
    I don't like how Nvidia's GPU auto-detect doesn't work on chrome.
  • -4 Hide
    thartist , July 7, 2011 5:47 AM
    Damn, Opera has it's flaws but it's nonetheless the one that does one thing best: browsing.
  • 9 Hide
    wheredahoodat , July 7, 2011 5:56 AM
    I have personally along others have remained loyal to Firefox for years, but the latest moves have been are puzzling. Despite the benchmarks, they have not addressed bad cold start up problems (test on netbooks not on desktop rigs to find out), and the Firefox 4/5 new javascript engine has been a nightmare memory wise causing many to banish Firefox or remain with version 3.6 on older computers.

    Also instead of prioritizing the electrolysis project which would have made Firefox as snappy as Chrome, they are wasting their time on the Azure graphics project to replace a only few months old Cairo engine for marginal benefit because html 5 is still just the future, with no real significant penetration for the web.

    Mozilla can only push their base so far with their incompetence. Back then there was no real competition or alternative for a big market share, standard compliance browser. However there is now a real alternative with Chrome, and Firefox's gradual but definite market share hemorrhaging has shown that incompetence has consequences.

  • 12 Hide
    beavermml , July 7, 2011 6:00 AM
    which one is the best in term of security out of box?
  • 2 Hide
    akorzan , July 7, 2011 6:05 AM
    On the last page in the table and row "proper page loads," why is Opera in twice and no Safari?
    Surely, this is a typo.
  • 1 Hide
    Maziar , July 7, 2011 6:19 AM
    Thanks for the review,I was waiting for this !
  • 12 Hide
    andy5174 , July 7, 2011 7:28 AM
    No doubt that Chrome is notably faster than Firefox so that I can feel the difference easily. However, Chrome lacks many useful features/add-ons offered by Firefox that I've been using for years. This is the main and only reason that Firefox is still my default browser.
  • 1 Hide
    obarthelemy , July 7, 2011 7:48 AM
    I'm still uncomfortable with those kind of tests. Especially because they don't take into account out-of-the-box features and creature comforts. I have all 4 installed (no Safari), and I find use Opera most, because it does very good Mouse Gestures, tab management, and synch. Without any addons ! I find Addons a very mixed blessing: I had plenty of headaches with FFox's, what with compatibility issues when upgrading FFox, bug and slow/ban/non-existent support... I find that having all required functionnality included in the base browser is a big plus, more-so for non-techies.
  • 3 Hide
    Anonymous , July 7, 2011 8:12 AM
    How can you place Firefox in Memory Usage as strong. This thing is leaking all over the place. Should you review your methodology?
    PS I m an early adopter of FF since 2.0 but I consider switching due to this memory issue.
  • 14 Hide
    johnsmithhatesVLC , July 7, 2011 8:13 AM
    Firefox is the only browser that can block all ads properly. It's also the most reliable at page loading as the WGP has proven. I really don't care about millisecond differences in page loads. Firefox is the best to me.
  • 4 Hide
    lucas1024 , July 7, 2011 8:45 AM
    Thanks for including the reliability test! I had almost convinced myself to switch to Chrome from FF5, because of the memory usage, but not anymore - I do like my pages to load every time and I routinely have 30-40 tabs open.
  • 0 Hide
    neiroatopelcc , July 7, 2011 8:49 AM
    page 3 :
    64-bit Desktop Test System
    Operating System
    Microsoft Windows 7 Ultimate (32-bit)
Display more comments