Today, Google and GitHub unveil the mysterious project RoboHornet. This ambitious all-in-one Web browser performance benchmark is like no other seen before, and we have early access! Will RoboHornet become the one browser benchmark to rule them all?
Everyone, say hello to RoboHornet, a new kind of Web browser performance benchmark. As reported by Fusible back in February, Google registered the robohornet.com and robohornet.org domains. Although it secured the benchmark's domain, RoboHornet is not a Google property; its an independent GitHub project.
But RoboHornet is so much more than just another browser benchmark. RoboHornet is a framework for a scalable, evolving performance metric. The “benchmark” is actually a modular suite of performance tests. These tests are created and voted on by a community of Web designers and developers, with standards boards and other Web heavyweights acting as stewards. We can confirm that Google, Facebook, Microsoft, and Mozilla are currently listed as stewards. The browser vendors and other Web companies can also get involved as technical advisers to the project. Even regular users can contribute to RoboHornet by raising issues, voting up existing issues, or even writing benchmarks.
Along with individual test results, RoboHornet outputs a single normalized score. So, this benchmark basically grades on a curve, meaning that even as the various browsers change and improve, they are still comparatively graded against each other, ensuring that RoboHornet will never max out. The weight given to each test in the final score is also decided by the community in order to account for current trends in Web design and development.
RoboHornet is only officially compatible with what most consider to be the “Top Five” browsers: Chrome, Firefox, Internet Explorer, Opera on Windows, and Safari on OS X. The status of Chrome, Firefox, and Opera on OS X and Linux is “untested.” The same goes for second-tier browsers like Maxthon, Sleipnir, and Camino. Also not making the cut are any Web browsers on Android and iOS, although making RoboHornet mobile-friendly was specifically mentioned as a high-priority future goal.
While the working status some Web browsers aren't specified officially, we'll put them all to the test on Windows 7, Windows 8 RTM, OS X Mountain Lion, Ubuntu 12.04 LTS, and even Android in the pages to come. But first, let's take the obligatory tour of our test setup.