Web Browser Grand Prix VI: Firefox 6, Chrome 13, Mac OS X Lion

Hardware And Test Setup

Hardware Setup

Swipe to scroll horizontally
Test System Specs
Operating System 1Microsoft Windows 7 Ultimate (64-bit)
Operating System 2Apple Mac OS X Lion (64-bit)
ProcessorIntel Core i5-750 (Lynnfield) @ 2.8 GHz, Quad-Core
MotherboardGigabyte GA-P55A-UD7, LGA 1156, P55 Express, F7 BIOS
Memory8 GB Crucial DDR3 @ 1333 MT/s (2 x 4 GB)
GraphicsAMD Radeon HD 4870 Reference Boards 512 GB GDDR5 (PCI-e 2.0)
StorageSeagate Barracuda 7200.12 500 GB SATA 3Gb/s, 7200 RPM, 16 MB Cache
OpticalAsus DRW-24B1ST/BLK/B/AS
Power SupplyCorsair TX750W (750 Watt Max)
ChassisZalman MS1000-HS2
CPU CoolerScythe Mugen 2 Revision B

The Windows 7 Test Installation

The OS X Lion Test Installation
Swipe to scroll horizontally
Local Web Server Specs
Operating SystemUbuntu 10.04 LTS Server Edition "Lucid Lynx" (32-bit)
ProcessorAMD Athlon @ 1150 MHz
MotherboardSoyo Dragon Platinum
Memory512 MB DDR
GraphicsAMD Radeon 9550, 256 MB GDDR
Storage40 GB Western Digital HDD WD400BB
OpticalSamsung DVD-ROM SD-616T
Extra PackagesApache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH

As requested by readers in our previous stories, the table below hosts additional information on the test network.

Swipe to scroll horizontally
Network Specs
ISP ServiceCox Premium (28 Mb/s down, 5 Mb/s up)
ModemMotorola SURFboard SBS101U
RouterLinksys WRT54G2 V1

Hackintosh Setup

The table below lists what we used to get the Lion-based Hackintosh up and running.

Swipe to scroll horizontally
InstallerxMove
BootloaderiBoot
DSDTGA-P55A-UD7

While these tests are conducted on Mac OS X, this test system is not an Apple-branded Mac. We had to use a different bootloader than what you'd find on a genuine Mac, and real Mac systems use EFI instead of BIOS. Therefore, performance may vary from the same tests conducted on an Apple-branded system.

Also keep in mind that this use of OS X is completely unauthorized, and anyone doing this at home should have no expectations of receiving any help or support from Apple.

Software Setup

Both our Windows 7 Ultimate and Mac OS X Lion installations were freshly installed and fully updated as of midnight on August 15th. Power management and automatic updates were disabled before testing. The Web browsers and additional software, along with the exact version numbers tested, are listed in the table below.

Swipe to scroll horizontally
SoftwareWindows VersionMac Version
Chrome13.0.782.21513.0.782.215
Firefox6.06.0
Internet Explorer9.0.8112.16421N/A
Opera11.50 (1074)11.50 (1074)
Safari5.1 (7534.50)5.1 (7534.48.3)
AMD Driver8.850.0.0ATI Radeon HD 4800 series
Adobe Flash10.3.183.510.3.183.5
Microsoft Silverlight4.0.60531.04.0.60531.0
Oracle Java6.0.26014.0.3

Test Setup

We restart the computer and allow it to idle for a while before benchmarking the next browser. Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variations.

This time around, we're rating the benchmarks themselves. The tests are placed into one of four groups: core, observation, dated, and quarantine.

Core tests are considered current. These tests are usually trusted industry standards or our own creations, and they make up the core of the WBGP suite. Examples include our own page-load time tests, FutureMark Peacekeeper, and Mozilla Dromaeo DOM.

Tests that are either generally unknown, mostly untested, or just too bleeding-edge are placed under observation. The WebGL tests and Ecma test262 are examples.

Tests classified as dated are either outdated, losing relevance, or otherwise need replacing. We are actively seeking community feedback and contributions regarding alternatives to these benchmarks. Examples of this group include Acid3 and GUIMark Java.

The final group is for quarantined benchmarks. Benchmarks find their way into quarantine by delivering dubious results or by being gamed. Examples include previous suite dropouts Google V8, Dromaeo JavaScript, and now SunSpider as well.

Whenever the results of benchmarks that test the same thing conflict, more weight is given to tests with a better rating when creating the analysis tables.

The table below lists all 41 of the tests currently in our test suite (along with a version number, where applicable), current rating in the Web Browser Grand Prix, and number of iterations performed:

Swipe to scroll horizontally
Web Browser Grand Prix Test Suite 6.0
Test NameIterationsRating
Performance Tests (32)
Startup Time: Single Tab5Core
Startup Time: Eight Tabs5Core
Page Load Time: Google5Core
Page Load Time: YouTube5Core
Page Load Time: Yahoo!5Core
Page Load Time: Amazon5Core
Page Load Time: Wikipedia5Core
Page Load Time: eBay5Core
Page Load Time: craigslist5Core
Page Load Time: The Huffington Post5Core
Page Load Time: Tom's Hardware5Core
Peacekeeper3Core
Kraken v1.13Core
SunSpider v0.9.13Quarantine
Dromaeo DOM3Core
Maze Solver5Core
JSGameBench v4.13Core
Asteroids HTML5 Canvas 2D And JavaScript3Observation
GUIMark 2 HTML5 Vector Charting (1 pixel variant)5Core
GUIMark 2 HTML5 Bitmap Gaming5Core
GUIMark 2 HTML5 Text Columns5Core
Psychedelic Browsing3Core
Hardware Acceleration Stress Test3Dated
WebGL FishIE5Observation
WebGL Solar System5Observation
ThoughtsInComputation Particles5Observation
GUIMark2 Flash Vector Charting5Core
GUIMark2 Flash Bitmap Gaming5Core
GUIMark2 Flash Text Columns5Core
Flash Benchmark 20083Core
GUIMark Java5Dated
Encog Silverlight5Dated
Efficiency Benchmarks (4)
Memory Usage: Single Tab3Core
Memory Usage: 40 Tabs3Core
Memory Management: -39 Tabs3Core
Memory Management: -39 Tabs (extra 5 minutes)3Core
Reliability Benchmarks (1)
Proper Page Loads3Core
Conformance Benchmarks (4)
HTML5Test.com1Core
CSS3 Selectors Test1Core
Ecma test2621Observation
Acid31Dated

You've seen the lineup and toured the track. Now it's off to the races.

  • ne0nguy
    The first chart says "higher is better" for the load time
    Reply
  • adamovera
    ne0nguyThe first chart says "higher is better" for the load timethank you, workin' on it
    Reply
  • SteelCity1981
    Chrome is the best browser out there right now. While FireFox maybe more popular then Chrome is, Chrome has shown why it is the best browser out today. If you haven't used Chrome yet it's def worth a look.
    Reply
  • soccerdocks
    The reader function in safari actually looks really nice. Although I'd never use Safari on principle. I hope other browsers implement a similar function.
    Reply
  • mayankleoboy1
    why does firefox(6/8/9) performa so horribly on the IE9 maze solover test?
    chrome13 completely obliterats it.

    and firefox 8/9 are still a memory hog.
    not really surprised by poor show of ie9. moat updates it gets are "security updates".
    Reply
  • tofu2go
    Being on a Macbook with only 3GB of memory, memory is the most important factor for me. I open a LOT of tabs and I keep them open for long periods. For awhile I used Chrome, but recently switched to Firefox 6 and saw my memory utilization drop by well over 1GB. Granted with Firefox I was able to do something I am not able to do in any other browser, I could group my tabs into tab groups. I believe this allows for more efficient memory management, i.e. only the current group uses much memory. Not having done any tests, this is pure speculation. All I know is that I'm seeing MUCH lower memory usage with Firefox on OSX. Despite what this article would suggest.
    Reply
  • @soccerdocks

    Yeah? And exactly what principle would that be?
    Reply
  • andy5174
    @Google:
    Bring back the Google Dictionary, otherwise I will use Bing Search, Firefox and Facebook instead of Google Search, Chrome and G+.
    Reply
  • kartu
    Firefox 6 comes in third for both OSes, representing a major drop from Firefox 5.
    According to the graphic on "Reliability Benchmarks: Proper Page Loads" on MacOS Firefox is actually second, not third.
    Reply
  • LaloFG
    I keep Opera, more memory used and time to load pages is nothing when it load pages correctly; and the feeling in its interface is the greater.
    Reply