Today, Google and GitHub unveil the mysterious project RoboHornet. This ambitious all-in-one Web browser performance benchmark is like no other seen before, and we have early access! Will RoboHornet become the one browser benchmark to rule them all?
Everyone, say hello to RoboHornet, a new kind of Web browser performance benchmark. As reported by Fusible back in February, Google registered the robohornet.com and robohornet.org domains. Although it secured the benchmark's domain, RoboHornet is not a Google property; its an independent GitHub project.
But RoboHornet is so much more than just another browser benchmark. RoboHornet is a framework for a scalable, evolving performance metric. The “benchmark” is actually a modular suite of performance tests. These tests are created and voted on by a community of Web designers and developers, with standards boards and other Web heavyweights acting as stewards. We can confirm that Google, Facebook, Microsoft, and Mozilla are currently listed as stewards. The browser vendors and other Web companies can also get involved as technical advisers to the project. Even regular users can contribute to RoboHornet by raising issues, voting up existing issues, or even writing benchmarks.

Along with individual test results, RoboHornet outputs a single normalized score. So, this benchmark basically grades on a curve, meaning that even as the various browsers change and improve, they are still comparatively graded against each other, ensuring that RoboHornet will never max out. The weight given to each test in the final score is also decided by the community in order to account for current trends in Web design and development.
Similar to early incarnations of our own Web Browser Grand Prix (WBGP), RoboHornet deals exclusively with performance testing. Today's Alpha 1 release includes CSS, DOM, HTML, and JavaScript. The individual tests are meant to mimic real-world scenarios that the community feels are integral for today's Web browsing experience. Basically, it seeks to test what actually matters to users. All the tests in RoboHornet are new, so this is not an amalgamation of existing industry standard metrics like our WBGP.
RoboHornet is only officially compatible with what most consider to be the “Top Five” browsers: Chrome, Firefox, Internet Explorer, Opera on Windows, and Safari on OS X. The status of Chrome, Firefox, and Opera on OS X and Linux is “untested.” The same goes for second-tier browsers like Maxthon, Sleipnir, and Camino. Also not making the cut are any Web browsers on Android and iOS, although making RoboHornet mobile-friendly was specifically mentioned as a high-priority future goal.
While the working status some Web browsers aren't specified officially, we'll put them all to the test on Windows 7, Windows 8 RTM, OS X Mountain Lion, Ubuntu 12.04 LTS, and even Android in the pages to come. But first, let's take the obligatory tour of our test setup.
They've got to quit bundling OS updates and IE together though. Otherwise, they get left behind when all the other browsers get updated, but IE doesn't. When IE 9 came out, it was quite the monster too, but 3 years is a long time for other browsers to catch and easily overtake it.
Didn't work. It may eventually finish but it spent way too long on the first test, pretty much the same story with Maxthon, Opera, and Sleipnir on Android. Same deal on iOS. They specifically mentioned that mobile was not yet supported, but it will be a priority in the future. Maxthon took a very long time on Windows, OS X, and Android, so I aborted it, but I have a feeling that it would finish eventually (could literally be hours though, and I didn't have time to wait). If anyone reports Maxthon, Sleipnir or Camino actually finishing the test, I'll give them another go and update the article accordingly.
2 out of 17 benchmark(s) failed.
My poor work PC
In normal use, IE10 performs worse than FF15, so i dont see how it scored that much better.
How exactly? I'm using IE10 for work and home and it's been superb thus far, with 0 crashes. Can't say the same for FF. It's a memory hog.
FWIW, the actual mozilla developers had no clue about this benchmark prior to this article. Infact, they have started working on this benchmark after someone reported this article. So IMO, the presence of Mozilla is marginal at best in the overseeing member committee
And, mozila has only one representative. More than half of the committee members are google people. Makes me wonder if there is some unintentional bias....
typed robohornet.com as the URL and get a friendly 404 error.
Actually have to use www.robohornet.com !
Now isn't that kinda sad ? :-)
http://www.robohornet.org/
BTW, FF15.0.1 automatically searched for .org when i typed the .com address, so no issues here
Time for you to check proxy settings ?
"I" obviously did not have problems figuring in out :-)
But the majority of casual surfers (i.e. my clients) will not bother after a 404 error. They hit reload and then give up and go somewhere else.
It's not that hard to configure a server to work with/and without www :-)
Things that don't make you go Hmmmmmmmm.
Just the other day another test was published, made by MS. In that test for all practical purposes IE and FFox were tied at #1. ahead of Chrome. http://www.favbrowser.com/internet-explorer-10-is-8-faster-than-google-chrome-20/
Robohornet do catch some specific slow spots in Firefox. But it also has some tests that look suspect. I expect both to improve, though. Look at the comments in this Bugzilla thread. https://bugzilla.mozilla.org/show_bug.cgi?id=793913
Because all of these programs were originally designed for Windows and "ported" to Mac.
1. Most browsers are single threaded. So quad core is meaningless.
2. This "1-2 secnd" difference is when loading simple pages. With complex pages using lots of CSS3 and HTML5, these "1-2second" differences become several 10's of seconds, which is noticable, unless you are "simple"
3.This benchmark also tests how responsive the browser is during a page load. If the browser freezes during a page load, any speed gain is quite meaningless.