Hardware And Test Setup
Hardware Setup
| Test System Specs | |
|---|---|
| Operating System 1 | Microsoft Windows 7 Ultimate (64-bit) |
| Operating System 2 | Apple Mac OS X Lion (64-bit) |
| Processor | Intel Core i5-750 (Lynnfield) @ 2.8 GHz, Quad-Core |
| Motherboard | Gigabyte GA-P55A-UD7, LGA 1156, P55 Express, F7 BIOS |
| Memory | 8 GB Crucial DDR3 @ 1333 MT/s (2 x 4 GB) |
| Graphics | AMD Radeon HD 4870 Reference Boards 512 GB GDDR5 (PCI-e 2.0) |
| Storage | Seagate Barracuda 7200.12 500 GB SATA 3Gb/s, 7200 RPM, 16 MB Cache |
| Optical | Asus DRW-24B1ST/BLK/B/AS |
| Power Supply | Corsair TX750W (750 Watt Max) |
| Chassis | Zalman MS1000-HS2 |
| CPU Cooler | Scythe Mugen 2 Revision B |
| Local Web Server Specs | |
|---|---|
| Operating System | Ubuntu 10.04 LTS Server Edition "Lucid Lynx" (32-bit) |
| Processor | AMD Athlon @ 1150 MHz |
| Motherboard | Soyo Dragon Platinum |
| Memory | 512 MB DDR |
| Graphics | AMD Radeon 9550, 256 MB GDDR |
| Storage | 40 GB Western Digital HDD WD400BB |
| Optical | Samsung DVD-ROM SD-616T |
| Extra Packages | Apache2, MySQL Client, MySQL Server, PHP5, PHP-GD, PHP5-MySQL, PHPMyAdmin, SSH |
As requested by readers in our previous stories, the table below hosts additional information on the test network.
| Network Specs | |
|---|---|
| ISP Service | Cox Premium (28 Mb/s down, 5 Mb/s up) |
| Modem | Motorola SURFboard SBS101U |
| Router | Linksys WRT54G2 V1 |
Hackintosh Setup
The table below lists what we used to get the Lion-based Hackintosh up and running.
| Installer | xMove |
|---|---|
| Bootloader | iBoot |
| DSDT | GA-P55A-UD7 |
While these tests are conducted on Mac OS X, this test system is not an Apple-branded Mac. We had to use a different bootloader than what you'd find on a genuine Mac, and real Mac systems use EFI instead of BIOS. Therefore, performance may vary from the same tests conducted on an Apple-branded system.
Also keep in mind that this use of OS X is completely unauthorized, and anyone doing this at home should have no expectations of receiving any help or support from Apple.
Software Setup
Both our Windows 7 Ultimate and Mac OS X Lion installations were freshly installed and fully updated as of midnight on August 15th. Power management and automatic updates were disabled before testing. The Web browsers and additional software, along with the exact version numbers tested, are listed in the table below.
| Software | Windows Version | Mac Version |
|---|---|---|
| Chrome | 13.0.782.215 | 13.0.782.215 |
| Firefox | 6.0 | 6.0 |
| Internet Explorer | 9.0.8112.16421 | N/A |
| Opera | 11.50 (1074) | 11.50 (1074) |
| Safari | 5.1 (7534.50) | 5.1 (7534.48.3) |
| AMD Driver | 8.850.0.0 | ATI Radeon HD 4800 series |
| Adobe Flash | 10.3.183.5 | 10.3.183.5 |
| Microsoft Silverlight | 4.0.60531.0 | 4.0.60531.0 |
| Oracle Java | 6.0.260 | 14.0.3 |
Test Setup
We restart the computer and allow it to idle for a while before benchmarking the next browser. Other than the conformance benchmarks, all of our final scores are an average of several iterations. More iterations are run on tests that have short durations, lower scales, and/or higher variations.
This time around, we're rating the benchmarks themselves. The tests are placed into one of four groups: core, observation, dated, and quarantine.
Core tests are considered current. These tests are usually trusted industry standards or our own creations, and they make up the core of the WBGP suite. Examples include our own page-load time tests, FutureMark Peacekeeper, and Mozilla Dromaeo DOM.
Tests that are either generally unknown, mostly untested, or just too bleeding-edge are placed under observation. The WebGL tests and Ecma test262 are examples.
Tests classified as dated are either outdated, losing relevance, or otherwise need replacing. We are actively seeking community feedback and contributions regarding alternatives to these benchmarks. Examples of this group include Acid3 and GUIMark Java.
The final group is for quarantined benchmarks. Benchmarks find their way into quarantine by delivering dubious results or by being gamed. Examples include previous suite dropouts Google V8, Dromaeo JavaScript, and now SunSpider as well.
Whenever the results of benchmarks that test the same thing conflict, more weight is given to tests with a better rating when creating the analysis tables.
The table below lists all 41 of the tests currently in our test suite (along with a version number, where applicable), current rating in the Web Browser Grand Prix, and number of iterations performed:
| Web Browser Grand Prix Test Suite 6.0 | ||
|---|---|---|
| Test Name | Iterations | Rating |
| Performance Tests (32) | ||
| Startup Time: Single Tab | 5 | Core |
| Startup Time: Eight Tabs | 5 | Core |
| Page Load Time: Google | 5 | Core |
| Page Load Time: YouTube | 5 | Core |
| Page Load Time: Yahoo! | 5 | Core |
| Page Load Time: Amazon | 5 | Core |
| Page Load Time: Wikipedia | 5 | Core |
| Page Load Time: eBay | 5 | Core |
| Page Load Time: craigslist | 5 | Core |
| Page Load Time: The Huffington Post | 5 | Core |
| Page Load Time: Tom's Hardware | 5 | Core |
| Peacekeeper | 3 | Core |
| Kraken v1.1 | 3 | Core |
| SunSpider v0.9.1 | 3 | Quarantine |
| Dromaeo DOM | 3 | Core |
| Maze Solver | 5 | Core |
| JSGameBench v4.1 | 3 | Core |
| Asteroids HTML5 Canvas 2D And JavaScript | 3 | Observation |
| GUIMark 2 HTML5 Vector Charting (1 pixel variant) | 5 | Core |
| GUIMark 2 HTML5 Bitmap Gaming | 5 | Core |
| GUIMark 2 HTML5 Text Columns | 5 | Core |
| Psychedelic Browsing | 3 | Core |
| Hardware Acceleration Stress Test | 3 | Dated |
| WebGL FishIE | 5 | Observation |
| WebGL Solar System | 5 | Observation |
| ThoughtsInComputation Particles | 5 | Observation |
| GUIMark2 Flash Vector Charting | 5 | Core |
| GUIMark2 Flash Bitmap Gaming | 5 | Core |
| GUIMark2 Flash Text Columns | 5 | Core |
| Flash Benchmark 2008 | 3 | Core |
| GUIMark Java | 5 | Dated |
| Encog Silverlight | 5 | Dated |
| Efficiency Benchmarks (4) | ||
| Memory Usage: Single Tab | 3 | Core |
| Memory Usage: 40 Tabs | 3 | Core |
| Memory Management: -39 Tabs | 3 | Core |
| Memory Management: -39 Tabs (extra 5 minutes) | 3 | Core |
| Reliability Benchmarks (1) | ||
| Proper Page Loads | 3 | Core |
| Conformance Benchmarks (4) | ||
| HTML5Test.com | 1 | Core |
| CSS3 Selectors Test | 1 | Core |
| Ecma test262 | 1 | Observation |
| Acid3 | 1 | Dated |
You've seen the lineup and toured the track. Now it's off to the races.
