RoboHornet: The Next Big Thing In Browser Benchmarking

Features
By
published

Today, Google and GitHub unveil the mysterious project RoboHornet. This ambitious all-in-one Web browser performance benchmark is like no other seen before, and we have early access! Will RoboHornet become the one browser benchmark to rule them all?

Test System Specs And Software Setup

Hardware Setup

Swipe to scroll horizontally
Desktop Test System Specs
Operating System 1Microsoft Windows 7 Ultimate (64-bit)

Operating System 2Microsoft Windows 8 Enterprise RTM (64-bit)

Operating System 3Apple OS X Mountain Lion 10.8.1 (64-bit)

Operating System 4Ubuntu 12.04.1 LTS Precise Pangolin (64-bit)

ProcessorIntel Core i5-2500K @ 3.3 GHz (quad-core)MotherboardGigabyte GA-Z68XP-UD3 (F10 BIOS)Memory8 GB Crucial DDR3 @ 1333 MT/s (2 x 4 GB)GraphicsGeForce GTX 560 Ti 1 GB GDDR5 (PCIe 2.0 x16)StorageSeagate Barracuda 7200.12 500 GB SATA 3Gb/s, 7200 RPM, 16 MB CacheOptical Asus DRW-24B1ST/BLK/B/ASPower SupplyCorsair TX750W (750 W max)CaseZalman MS-1000 HS2CPU CoolerScythe Mugen 2 Revision BKeyboardLogitech Wireless Keyboard K320MouseLogitech Wireless Trackball M570

We used a fully updated Asus Transformer TF300T tablet running the latest version of Android Jelly Bean (4.1.1) for the Android tests.

Swipe to scroll horizontally
Tablet Test System Specs
ModelAsus Transformer TF300T
Operating SystemGoogle Android 4.1.1 (Jelly Bean)

Display10.1" LED-Backlit WXGA IPS (1280x800)ProcessorNvidia Tegra 3 (quad-core)Memory1 GBStorage32 GBNetworkWLAN 802.11 b/g/n @ 2.4 GHz

The table below holds additional information on the test network.

Swipe to scroll horizontally
Network Specs
ISP ServiceCox Preferred (18 Mb/s down, 2 Mb/s up)
ModemMotorola SURFboard SBS101U
RouterLinksys WRT54G2 V1
Image 1 of 2

Software Setup

The Web browsers, operating systems, drivers, and other supporting apps were fully updated as of midnight on September 24, 2012 (Eastern Daylight Time). Screensaver and power management options are disabled in each operating system before testing.

All the software we installed, including the exact version number of the browsers tested, is listed in the table below.

Swipe to scroll horizontally
SoftwareVersion
Chrome21.0.1180.89 m
Firefox15.0.1
Internet Explorer 99.0.8112.16421
Internet Explorer 9 (64-bit)9.0.8112.16421 64-bit Edition
Internet Explorer 10 RTM10.0.9200.16384
Opera12.02 (build 1578)
Safari6.0.1 (8536.26.14)
Nvidia Driver (Windows-only)306.23

Methodology

Per the RoboHornet homepage, each browser is tested with no other tabs or applications open. Due to the long duration of this benchmark and the short runway we had prior to publication, the charts contain the result of just one iteration. However, we perform a second, unrecorded iteration in order to verify that the first result is repeatable within acceptable variance.

Adam Overa
26 Comments Comment from the forums
  • aicom
    Nice to see that IE10 has got some fight in it. I haven't really used it much on Win8 after installing Chrome though. If IE can get that plugin ecosystem that Firefox and Chrome have plus some of the nice cloud syncing stuff with Win8/RT/Phone that Safari and Chrome do with their mobile brethren, I'd definitely consider IE.

    They've got to quit bundling OS updates and IE together though. Otherwise, they get left behind when all the other browsers get updated, but IE doesn't. When IE 9 came out, it was quite the monster too, but 3 years is a long time for other browsers to catch and easily overtake it.
    Reply
  • puddleglum
    Good to see the browsers broken out by OS. Not everyone runs MS.
    Reply
  • How about firefox on android ?
    Reply
  • adamovera
    hheexxHow about firefox on android ?Didn't work. It may eventually finish but it spent way too long on the first test, pretty much the same story with Maxthon, Opera, and Sleipnir on Android. Same deal on iOS. They specifically mentioned that mobile was not yet supported, but it will be a priority in the future. Maxthon took a very long time on Windows, OS X, and Android, so I aborted it, but I have a feeling that it would finish eventually (could literally be hours though, and I didn't have time to wait). If anyone reports Maxthon, Sleipnir or Camino actually finishing the test, I'll give them another go and update the article accordingly.
    Reply
  • aaab
    RH-A1:0092.97
    2 out of 17 benchmark(s) failed.

    My poor work PC :(
    Reply
  • What an awful collection of microbenchmarks. Go read Hennessy and Patterson; the best benchmarks are real apps, not tiny little tests that measure a single thing.
    Reply
  • mayankleoboy1
    IE10's win on Win8 makes me wonder if the benchmark makes use of specialised data arrays, which are supported in IE10 only.

    In normal use, IE10 performs worse than FF15, so i dont see how it scored that much better.
    Reply
  • alikum
    mayankleoboy1IE10's win on Win8 makes me wonder if the benchmark makes use of specialised data arrays, which are supported in IE10 only.In normal use, IE10 performs worse than FF15, so i dont see how it scored that much better.How exactly? I'm using IE10 for work and home and it's been superb thus far, with 0 crashes. Can't say the same for FF. It's a memory hog.
    Reply
  • mayankleoboy1
    ^ to each his own :)

    FWIW, the actual mozilla developers had no clue about this benchmark prior to this article. Infact, they have started working on this benchmark after someone reported this article. So IMO, the presence of Mozilla is marginal at best in the overseeing member committee

    And, mozila has only one representative. More than half of the committee members are google people. Makes me wonder if there is some unintentional bias....
    Reply
  • freggo
    Win 7 with Firefox 15.0.1

    typed robohornet.com as the URL and get a friendly 404 error.
    Actually have to use www.robohornet.com !

    Now isn't that kinda sad ? :-)

    Reply