Sign in with
Sign up | Sign in

Test Setup And Benchmark Suite

Which Web Browser Should You Run On Your Android Device?
By

Versus The Desktop Web Browser Grand Prix

As we stated in our exploration of iOS-based Web browser performance, much of the desktop benchmark suite had to be dropped from mobile testing. Let's quickly recap the losses.

Start time is scaled back to just a single-tab “hot” start. Mobile OSes only load the active tab, and background tabs aren't loaded until they are made active. This makes the eight-tab start time irrelevant. Since tablets are usually never powered down unless they run out of battery, “cold” start times are also irrelevant.

Current mobile operating systems handle memory and multi-tasking in a very different manner than PCs, so the memory efficiency tests from the desktop are not needed, either. With the loss of the 40-tab memory test and the inability to load all tabs simultaneously, the page load reliability test cannot be conducted either.

We also lost plug-ins, so Flash, Java, and Silverlight aren't needed. WebGL was jettisoned too, since no mobile Web browser supports it yet.

Versus The iOS Web Browser Grand Prix

We lost WebVizBench and Psychedelic Browsing on Android, since the tests fail to run on too many browsers. However, JSGameBench makes a return as our hardware acceleration benchmark.

There was no need to substitute the GUIMark2 HTML5 Vector Charting tests with GUIMark3's Vector Test, as we did on the iPad. In an odd reversal, the GUIMark3 version crashes on most Android browsers. We did, however, need to scrap the Bitmap Gaming portion of the test, as well as the Asteroids HTML5 Canvas 2D & JavaScript benchmark because Dolphin has issues running those tests.

Sadly, we're all out of CSS performance tests: Kaizoumark will not run on several browsers, the CSS Stress Test & Performance Profiling is a bookmarklet and will therefore not work on Android, and Maze Solver still has serious issues with Firefox.

Test Setup And Benchmark Suite

Test System Specs
Model
Asus Transformer TF300T
Operating System
Google Android 4.1.1 "Jelly Bean"Home ScreenHome Screen
Display
10.1" LED-Backlit WXGA IPS (1200x800)
Processor
Nvidia Tegra 3 (quad-core) @ 1.2 GHz
Graphics
Nvidia ULP GeForce
Memory
1 GB DDR3
Storage
32 GB eMMC Flash
Networking
802.11 b/g/n @ 2.4 GHz
Local Web Server Specs
Operating System
Ubuntu 12.04.1 LTS Server Edition "Precise Pangolin" (32-bit)
Processor
Intel Pentium 4 @ 2.41 GHz
Motherboard
Biostar P4M80-M4
Memory
768 MB DDR @ 333 MHz
Graphics
Nvidia GeForce FX 5500 128 MB DDR (AGP)
Storage
Western Digital Caviar SE WD1600AAJD, 160 GB EIDE, 7200 RPM
Extra Packages
Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH, Node.js, NPM
Network Specs
ISP Service
Cox Preferred (18 Mb/s down, 2 Mb/s up)
Modem
Arris Touchstone Telephony Modem TM502G
Router
Linksys WRT54G2 V1
Benchmark Suite
Page Load Time
Start Time (Yahoo! homepage, Cached)
Uncached Load Time (Eight Test Pages)
Cached Load Time (Eight Test Pages)
JavaScript
RIABench JavaScript (Eight Tests)
Apple SunSpider v0.9.1 (Google Mod)
Mozilla Kraken v1.1
Google Octane v1
FutureMark Peacekeeper v2
RightWare BrowserMark v2
DOM
Mozilla Dromaeo DOM Core
HTML5
GUIMark2 HTML5 (Three Tests)
HTML5 Canvas Performance Test
Impact HTML5 Benchmark
Mandelbrot Set in HTML5
Hardware Acceleration
Facebook JSGameBench v0.4.1
Standards Conformance
HTML5Test.com
The CSS3 Test
Ecmascript Language test262
Facebook ringmark
Security
Browserscope Security

Although applicable links are included in the table above, we've also created a delicious account dedicated to chronicling links to Web Browser Grand Prix benchmarks.

The Transformer, local Web server, and all software were updated as of midnight (Eastern Standard Time) on November 24, 2012. Detailed methodologies are explained on the individual benchmark pages.

Display all 65 comments.
This thread is closed for comments
Top Comments
  • 14 Hide
    aznshinobi , November 30, 2012 3:59 AM
    Quote:
    "Stock Android Browser" is a myth. There is NO "Stock" android browser.
    Each device manufacturer (Samsung, Asus, Lg, HTC) customise/modify the "stock" browser to match the SoC, the TDP, power saving, and specific browser benchmark targeted, for that device.

    So this "Stock" browser is actually a modified browser, customised by ASUS to work better with a Tegra3 SoC, in some specifc benchmarks which Asus thinks are more important than others. Its not a representative of all android devices.


    When you're running a Nexus device, it's a stock browser...
Other Comments
  • -5 Hide
    mayankleoboy1 , November 30, 2012 3:36 AM
    "Stock Android Browser" is a myth. There is NO "Stock" android browser.
    Each device manufacturer (Samsung, Asus, Lg, HTC) customise/modify the "stock" browser to match the SoC, the TDP, power saving, and specific browser benchmark targeted, for that device.

    So this "Stock" browser is actually a modified browser, customised by ASUS to work better with a Tegra3 SoC, in some specifc benchmarks which Asus thinks are more important than others. Its not a representative of all android devices.
  • 3 Hide
    mayankleoboy1 , November 30, 2012 3:37 AM
    Sunspider and Kraken are crap benchmarks. All browsers target these benchmarks for specifc optimisations, that are never actually used on the web.
  • 14 Hide
    aznshinobi , November 30, 2012 3:59 AM
    Quote:
    "Stock Android Browser" is a myth. There is NO "Stock" android browser.
    Each device manufacturer (Samsung, Asus, Lg, HTC) customise/modify the "stock" browser to match the SoC, the TDP, power saving, and specific browser benchmark targeted, for that device.

    So this "Stock" browser is actually a modified browser, customised by ASUS to work better with a Tegra3 SoC, in some specifc benchmarks which Asus thinks are more important than others. Its not a representative of all android devices.


    When you're running a Nexus device, it's a stock browser...
  • 4 Hide
    adamovera , November 30, 2012 4:02 AM
    mayankleoboy1Sunspider and Kraken are crap benchmarks. All browsers target these benchmarks for specifc optimisations, that are never actually used on the web.

    SunSpider is the next to go for sure, but I haven't heard a ton of criticism regarding Kraken yet. Between BrowserMark, Peacekeeper, and RIABench, we could withdraw all the vendor-developed JS tests.
  • 7 Hide
    tiret , November 30, 2012 4:57 AM
    give me a browser with flash support then we'll talk
  • 4 Hide
    mayankleoboy1 , November 30, 2012 5:03 AM
    ^ coming soon to firefox. Project Shumway.
  • 3 Hide
    tiret , November 30, 2012 5:10 AM
    ^ interesting. lets hope it works out... my gf is rather pissed that she can't play farmville on my galaxy tab.
  • 6 Hide
    fwupow , November 30, 2012 5:33 AM
    I've already figured out that Chrome isn't so hot, but the reason why Chrome still wins for me is that it synchronizes bookmarks, passwords, history and a bunch of other stuff across all my computers and devices. That is an indispensable feature for me.
  • 0 Hide
    wildkitten , November 30, 2012 6:56 AM
    tiretgive me a browser with flash support then we'll talk

    Since Adobe themselves has ended Flash development for all mobile platforms, I don't think you will see many browsers keeping support for it for long. Likely in a year, maybe 18 months, you won't see any support for Flash as, well, what's the point.
  • 7 Hide
    Anonymous , November 30, 2012 7:01 AM
    Firefox Beta has flash support once you download and install the flash apk - I have it working well on my Nexus 7
  • 8 Hide
    Anonymous , November 30, 2012 7:14 AM
    It is not benchmark results that counts when choosing my mobile browser. And what counts?
    - some kind of adblocking utility, like urlfilter.ini in Opera, adblock in firefox
    - minimum font size. On my 8" 160dpi screen fonts are too small to be read comfortably. In Opera I can declare a minimum font size and have all webpages readable and well formatted.
    - declare to be a desktop browser. I most browsers I have to declare this for every site I visit, in Opera I do it once and it is set
    - smoothness on any hardware, not only Quad-core. Try stock browser or firefox on a single-core A10 machine and it will freeze for 10s when composing your webpage. Opera won't be butter-smooth, but it will let you scroll througn the page from the beginning.
    So Opera is the worst browser according to Thg, but is the best for me, the average user.
  • 1 Hide
    Tomtompiper , November 30, 2012 7:15 AM
    Read this on Dolphin on my Galaxy Tab, Dolphin has been a fixture on the Tab for two years and for good reason. Forget the scores, even if it wasn't top I would still use it, the actual user experience is head and shoulders above the competition.
  • 4 Hide
    obarthelemy , November 30, 2012 8:23 AM
    piotrjakubIt is not benchmark results that counts when choosing my mobile browser. And what counts?- some kind of adblocking utility, like urlfilter.ini in Opera, adblock in firefox- minimum font size. On my 8" 160dpi screen fonts are too small to be read comfortably. In Opera I can declare a minimum font size and have all webpages readable and well formatted.- declare to be a desktop browser. I most browsers I have to declare this for every site I visit, in Opera I do it once and it is set- smoothness on any hardware, not only Quad-core. Try stock browser or firefox on a single-core A10 machine and it will freeze for 10s when composing your webpage. Opera won't be butter-smooth, but it will let you scroll througn the page from the beginning.So Opera is the worst browser according to Thg, but is the best for me, the average user.


    This. A review of performance only is useless. Features are more important.
  • 1 Hide
    obarthelemy , November 30, 2012 8:24 AM
    Also, Opera has a night mode that works, the only browser I could find with one. It's a little hacky though (http://forum.xda-developers.com/showthread.php?t=1326434)
  • 2 Hide
    gilahacker , November 30, 2012 8:35 AM
    "Boat Browser" (https://play.google.com/store/apps/details?id=com.boatbrowser.free) supports flash just fine. I've used it on a Galaxy S3 and a Nexus 10. Just go download the latest Flash apk from Adobe:

    http://helpx.adobe.com/flash-player/kb/archived-flash-player-versions.html#main_Archived_Flash_Player_versions_for_developers

    Scroll down a bit to get to the Android listings.
  • 2 Hide
    Anonymous , November 30, 2012 10:13 AM
    naked browser

    the ui is very minimal though so might not be newcomer friendly
    but the responsiveness is on par with dolphin
  • 1 Hide
    swyn01 , November 30, 2012 10:51 AM
    Chrome is the default browser for Nexus devices running Jellybean. Browser is not even installed on neither the Galaxy Nexus nor the Nexus 4.
  • 1 Hide
    Anonymous , November 30, 2012 11:30 AM
    Chrome is not only the default for Nexus devices...Razr M, HD, and Maxx HD also come with Chrome as the only browser installed. It's been great for me and I especially like being able to swipe between tabs.
  • 5 Hide
    Anonymous , November 30, 2012 11:59 AM
    so, according to real life browsing opera has fastest browsing, but because it scores last in syntethic benchmarks it comes last?
  • -2 Hide
    Mygind , November 30, 2012 1:06 PM
    I've had the TF300T for some month using Dolphin... And It is still slow, slow, slow.

    My next buy will be a labtop Core i5 / HD 4000 with a SSD and a mouse with a wire. It will be set up so it is always on standby - and wakes up in 2-4 sec.

    Tablets are for todlers and teens :-)
Display more comments