Reliability Benchmarks: Proper Page Loads
The memory usage benchmarks have been part of the Grand Prix since WBGP2, and during the 40-tab test we get to observe several strange behaviors from the various Web browsers. Until now, these were merely observations, not something we specifically looked for. But the inability of Safari to load all 40 tabs at the same time and the need to reload several pages in Chrome fall into the category of reliability. Beginning with this competition, the WBGP now sports reliability benchmarks.
We jot down every page that needs a refresh due to broken or missing elements from the 40-tab memory usage test in order to get a sense of how reliably each browser loads Web pages under heavy load. Obviously, all of the contenders properly load a single page, so a more extreme scenario is really the only way these failures manifest themselves consistently. Examples of page load failures include: error pages, blank pages, missing/broken images, broken layouts, frozen/missing videos, and blank ads.
The chart below lists the average number of tabs that required a refresh due to broken or missing elements.
As you can see, Firefox deserves its reputation as the rock-steady browser. Mozilla's latest creation succeeds at loading every single element of every single Web page in all 40 tabs, all at once, every single time. This is something that we've noticed throughout the WBGP. But this time we recorded it. Opera ties for first, also requiring zero reloads during the three iterations of the 40-tab memory usage tests. Google Chrome is consistently one of the fastest Web browsers in the pack, yet it requires multiple reloads due to broken or unfinished pages under the exact same load. This time around, we recorded 5.333 reloads on average. While not the greatest for reliability, this is definitely an improvement over recent versions.
The chart shows that Microsoft Internet Explorer 9 required an average of 13.666 reloads, however, the story is actually much worse than what we could record. IE9 often required multiple reloads per tab, meaning that after all 40 tabs were loaded and we reloaded the broken ones, some of them remained broken. We consistently had trouble loading YouTube, Wikipedia, The Huffington Post, and C3 Metrics with Internet Explorer. There were many other broken sites during each iteration, but these four pages needed to be reloaded multiple times on each of the three iterations. IE9 also crashed after the initial load requiring the application to be restored on each iteration. Strangely enough, IE9 exhibits the bizarre behavior of loading simple white space-heavy Web sites like Wikipedia and Craigslist in all black.
While the chart shows that Safari only needs an average of one reload per iteration, its placement at the bottom of the chart is no mistake. Remember that Safari simply will not finish loading all 40 tabs at the same time, and that we have to open each site individually in a new tab. This fact alone puts Safari at the bottom. The single reload average is from loading sites individually.
So there you have it. For proper page loads, you just can't beat Mozilla Firefox or Opera. Speed demons Chrome and IE9 get the job done first, but the results are often sloppy; much more so in the case of IE9. Apple's Safari simply cannot compete on this field. For power users, the ability to properly load a ton of pages is what separates the men from the boys, and right now Firefox 5 and Opera 11.50 are Chuck Norris.
