Sign in with
Sign up | Sign in

Is RoboHornet The Holy Grail Of Browser Benchmarks?

RoboHornet: The Next Big Thing In Browser Benchmarking
By

RoboHornet Alpha1: Initial Results

Now we've truly seen it all. Today, Google helped launch a modular, independent, and open source browser benchmark made up of tests created and voted on by Web developers and designers, with consultation from standards bodies and vendors alike.

Chrome is still the top choice under Linux and Windows 7, and Safari is still king in Mountain Lion. Opera is somewhere in the middle of the pack on Windows and Ubuntu, but lands at the bottom of the heap under OS X. In a surprise upset, Firefox falls flat on its face on every platform except OS X, where it manages to place ahead of Opera. But the real shocker happens on Microsoft's upcoming Windows 8, where the RTM build of Internet Explorer 10 comes out swinging, earning far and away the best score in this new test.


With the exception of the Android tests, we used our usual Web Browser Grand Prix test system. Due to the limited window between gaining access to RoboHornet and publishing, that's the only combination of hardware we could benchmark in time. It should be live and publicly accessible by now, so try it out on your own rig and let us know the results in the comments section. We should note that we also attempted to run RoboHornet on the new iPad, but Safari (and more than likely any other iOS browser) would not get past the first test.

How does RoboHornet affect the Web Browser Grand Prix?

For now, it doesn't. We were told that it would be inappropriate to include RoboHornet in our regular browser benchmarks in its current state. However, we'll most definitely be keeping a close eye on this benchmark for inclusion into the Web Browser Grand Prix. Perhaps one day it can even replace the majority of our performance-oriented metrics. Needless to say, we're very stoked about the future of RoboHornet."

Until then, the next couple articles in the Web Browser Grand Prix are, for the time being, outside of RoboHornet's core competency: Android and iOS. Stay tuned.

Follow Adam on Twitter.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 26 comments.
This thread is closed for comments
  • 8 Hide
    aicom , September 24, 2012 9:26 PM
    Nice to see that IE10 has got some fight in it. I haven't really used it much on Win8 after installing Chrome though. If IE can get that plugin ecosystem that Firefox and Chrome have plus some of the nice cloud syncing stuff with Win8/RT/Phone that Safari and Chrome do with their mobile brethren, I'd definitely consider IE.

    They've got to quit bundling OS updates and IE together though. Otherwise, they get left behind when all the other browsers get updated, but IE doesn't. When IE 9 came out, it was quite the monster too, but 3 years is a long time for other browsers to catch and easily overtake it.
  • 2 Hide
    puddleglum , September 24, 2012 9:31 PM
    Good to see the browsers broken out by OS. Not everyone runs MS.
  • -1 Hide
    Anonymous , September 24, 2012 10:01 PM
    How about firefox on android ?
  • 4 Hide
    adamovera , September 24, 2012 10:38 PM
    hheexxHow about firefox on android ?

    Didn't work. It may eventually finish but it spent way too long on the first test, pretty much the same story with Maxthon, Opera, and Sleipnir on Android. Same deal on iOS. They specifically mentioned that mobile was not yet supported, but it will be a priority in the future. Maxthon took a very long time on Windows, OS X, and Android, so I aborted it, but I have a feeling that it would finish eventually (could literally be hours though, and I didn't have time to wait). If anyone reports Maxthon, Sleipnir or Camino actually finishing the test, I'll give them another go and update the article accordingly.
  • 1 Hide
    aaab , September 24, 2012 11:45 PM
    RH-A1:0092.97
    2 out of 17 benchmark(s) failed.

    My poor work PC :( 
  • -5 Hide
    Anonymous , September 25, 2012 12:24 AM
    What an awful collection of microbenchmarks. Go read Hennessy and Patterson; the best benchmarks are real apps, not tiny little tests that measure a single thing.
  • -3 Hide
    mayankleoboy1 , September 25, 2012 2:20 AM
    IE10's win on Win8 makes me wonder if the benchmark makes use of specialised data arrays, which are supported in IE10 only.

    In normal use, IE10 performs worse than FF15, so i dont see how it scored that much better.
  • 3 Hide
    alikum , September 25, 2012 2:46 AM
    mayankleoboy1IE10's win on Win8 makes me wonder if the benchmark makes use of specialised data arrays, which are supported in IE10 only.In normal use, IE10 performs worse than FF15, so i dont see how it scored that much better.

    How exactly? I'm using IE10 for work and home and it's been superb thus far, with 0 crashes. Can't say the same for FF. It's a memory hog.
  • 0 Hide
    mayankleoboy1 , September 25, 2012 3:12 AM
    ^ to each his own :) 

    FWIW, the actual mozilla developers had no clue about this benchmark prior to this article. Infact, they have started working on this benchmark after someone reported this article. So IMO, the presence of Mozilla is marginal at best in the overseeing member committee

    And, mozila has only one representative. More than half of the committee members are google people. Makes me wonder if there is some unintentional bias....
  • 3 Hide
    freggo , September 25, 2012 4:03 AM
    Win 7 with Firefox 15.0.1

    typed robohornet.com as the URL and get a friendly 404 error.
    Actually have to use www.robohornet.com !

    Now isn't that kinda sad ? :-)

  • 2 Hide
    mayankleoboy1 , September 25, 2012 4:14 AM
    ^ the correct site is :

    http://www.robohornet.org/

    BTW, FF15.0.1 automatically searched for .org when i typed the .com address, so no issues here :) 
    Time for you to check proxy settings ?
  • 2 Hide
    freggo , September 25, 2012 4:38 AM
    mayankleoboy1^ the correct site is : http://www.robohornet.org/BTW, FF15.0.1 automatically searched for .org when i typed the .com address, so no issues here Time for you to check proxy settings ?


    "I" obviously did not have problems figuring in out :-)
    But the majority of casual surfers (i.e. my clients) will not bother after a 404 error. They hit reload and then give up and go somewhere else.

    It's not that hard to configure a server to work with/and without www :-)


  • 1 Hide
    davewolfgang , September 25, 2012 10:31 AM
    Wow....a bunch of "Tests" by the makers of ONE of the browsers that - OMG - the main competition loses everything, and doesn't even show on some.

    Things that don't make you go Hmmmmmmmm.
  • 2 Hide
    Anonymous , September 25, 2012 11:02 AM
    "Today, Google helped launch a modular, independent, and open source browser benchmark": This is literally not true, Google is the supplier of Chrome, and therefor not independent.
  • 0 Hide
    Anonymous , September 25, 2012 2:15 PM
    It is of course way too early to use this benchmark as an indication about real world browser performance.

    Just the other day another test was published, made by MS. In that test for all practical purposes IE and FFox were tied at #1. ahead of Chrome. http://www.favbrowser.com/internet-explorer-10-is-8-faster-than-google-chrome-20/

    Robohornet do catch some specific slow spots in Firefox. But it also has some tests that look suspect. I expect both to improve, though. Look at the comments in this Bugzilla thread. https://bugzilla.mozilla.org/show_bug.cgi?id=793913
  • -1 Hide
    Avro Arrow , September 25, 2012 2:57 PM
    These numbers seem big and meaningful but remember, we're talking a MAXIMUM DIFFERENCE of 1 (maybe 2) seconds between the "top" browser and the "bottom" browser in page load time. This is not something that is going to make or break your internet experience, especially considering that most of us on tomshardware are enthusiasts that have powerful rigs with Quad-Core or better CPUs running at 3GHz or more. Pick the browser you like most and use it! The feature set and feel of a browser are a far bigger issue to me on my computer (desktop or laptop) than speed. Speed affects smartphones far more and from my own personal android experience, stability is key, not speed. In that respect, I've found that Opera for android is the best of that bunch. As for Apple anything, I wouldn't know. I'm allergic to "Apples"... :sol
  • -1 Hide
    bwric , September 25, 2012 4:04 PM
    How come the Windows results are double that of the Mac?
  • 1 Hide
    Avro Arrow , September 26, 2012 2:55 AM
    bwricHow come the Windows results are double that of the Mac?

    Because all of these programs were originally designed for Windows and "ported" to Mac.
  • -1 Hide
    mayankleoboy1 , September 26, 2012 6:02 AM
    Avro ArrowThese numbers seem big and meaningful but remember, we're talking a MAXIMUM DIFFERENCE of 1 (maybe 2) seconds between the "top" browser and the "bottom" browser in page load time. This is not something that is going to make or break your internet experience, especially considering that most of us on tomshardware are enthusiasts that have powerful rigs with Quad-Core or better CPUs running at 3GHz or more. Pick the browser you like most and use it! The feature set and feel of a browser are a far bigger issue to me on my computer (desktop or laptop) than speed. Speed affects smartphones far more and from my own personal android experience, stability is key, not speed. In that respect, I've found that Opera for android is the best of that bunch. As for Apple anything, I wouldn't know. I'm allergic to "Apples"... :sol


    1. Most browsers are single threaded. So quad core is meaningless.

    2. This "1-2 secnd" difference is when loading simple pages. With complex pages using lots of CSS3 and HTML5, these "1-2second" differences become several 10's of seconds, which is noticable, unless you are "simple"

    3.This benchmark also tests how responsive the browser is during a page load. If the browser freezes during a page load, any speed gain is quite meaningless.
  • 1 Hide
    ojas , September 26, 2012 6:04 AM
    hmmm. i'm getting 118 on chrome and 82 with IE9x64
Display more comments