Sign in with
Sign up | Sign in

IE9 Beta: Better Is Not Enough

By - Source: Tom's Hardware US | B 63 comments
Tags :

I hate to crash Microsoft IE9 Beta party early, but it appears that the browser will miss its mark.

There is no doubt that IE9 will be a much better browser than IE8 and Microsoft is absolutely heading into the right direction, but IE9 does not deliver what I hoped it would. The performance we have seen so far is mediocre at best and the screenshots that were leaked make IE9 look like a half-baked product. Is this the best Microsoft can do or will the company surprise us on September 15 with a truly innovative browser?

Last week we finally were informed that Microsoft will be releasing its new IE9 Beta during a fancy September party in San Francisco, at which the company will pitch the IE9 story to a select group of bloggers, developers and journalists. It somewhat reminded me at the good old Microsoft times back in the mid-90s, when the company discovered just how appealing beta software is to users – Microsoft gave out 1000 free beta copies of Windows 95 through a prize drawing at the magazine I worked back then and threw a huge bash when the IE4 Beta was launched. If you were not around back then, IE4 was the browser Microsoft designed to bury Netscape and it was, conceivably, Microsoft’s big bet on proprietary web technologies as HTML 4 was introduced.

13 years later, we are looking at version 9 that is Microsoft’s first HTML 5 capable web browser. Its step up from IE8 is about as significant as the step from IE3 to IE4 and Microsoft is yet again in a scenario of a browser war. This time, however, it needs to defend its market leadership against the mighty Google, which is taking aim at Microsoft in most of its core markets. Also, today’s browser war is much more competitive than what we have seen in the mid- to late-90s – there is Mozilla, Apple and we should not forget Opera. Microsoft’s browsers have been bleeding market share for several years and have lost more than 35 points of share so far – from about 95% to about 60%. Microsoft recently gained some points back, but we should be realistic and see that this trend may be short-lived as it is due to an advertising campaign Microsoft has launched across multiple TV channels. As soon as the campaign stops, Microsoft’s share is likely to drop again.

But what about IE9? What if this next browser is as great as IE4 was? The enthusiasm is certainly there. Microsoft is bold again and has some great technologies in this new browser, a new JavaScript engine and perhaps a few more surprises we do not know about yet. If we go by Microsoft’s IE blog, then the browser has taken over the world already, almost. According to Microsoft’s benchmarks, the latest IE9 PP4 is already faster than Firefox (3.6.x and Firefox 4 Beta) and Safari 5. The chart indicates that the browser is almost as fast as Chrome 5, Chrome 6 and Opera. To me, such charts are always an invitation to run the benchmarks myself to check if those numbers are accurate. The claims are rather bold and I was wondering whether IE9 has made such huge progress or whether Microsoft’s IE9 team is pulling an Apple here and tries to get away with something that may not be exactly true. And, most importantly, we should get a good idea how good IE9 Beta may be when it is released next month. Let’s have a look.

Benchmarks: V8, Sunspider

Benchmarking browsers is always a difficult exercise. Browser performance depends on many variables we can’t stabilize and there are different benchmarks that play into different strengths. For example, V8 is a browser that caters to Google’s Chrome browser, Microsoft’s HTML5 tests let IE9 shine and Mozilla’s Dromaeo suite typically favors Firefox strengths. However, Webkit’s Sunspider is cited by all browser manufacturers as the most important performance indicator (JavaScript) these days and I personally like to run new browsers through Google’s V8 as it highlights performance changes between different browser versions. However, comparing different browsers to Chrome in V8 is a bit unfair.

Also, please note that the results are highly dependent on computer hardware. I ran the tests using a fairly antiquated Q6600 Intel quad-core CPU, 8 GB of memory, and an ATI HD 5570 graphics card.

Here is the result for Google’s V8 benchmark:

The Webkit browsers Chrome and Safari perform well here, but it is worth noting just how much Chrome is different from Safari and how much fine tuning has gone into Chrome. Also note Opera’s excellent result. As for IE9, Microsoft promised fine tuning or its JavaScript engine when it released IE9 PP1 and it certainly delivered as the browser nearly doubled its performance in V8. Notice that the latest IE9 PP4 is about 12x faster here than IE8. The same can be said about Firefox, however. The TraceMonkey (TM) JavaScript engine is significantly faster in Firefox 4.0 Beta 3 than it is in the current stable 3.6.8. The differences between Firefox and IE9 are almost negligible at this point, which is somewhat interesting as Microsoft pretty much forgets about Firefox in its benchmarks and blog posts already and has been focusing on Chrome instead.

I would expect IE9 Beta to gain more performance over PP4, but then we know that Mozilla is already preparing the launch of its new JaegerMonkey (JM) JavaScript engine as well, most likely with Firefox 4 Beta 5, which is due in early September. We have no exact performance numbers for JM at this point, but we hear that JM is about 15% faster than TM right now and Mozilla expects further gains. If we believe the developer team, JM’s V8 performance should be at about 1300 points, which puts it ahead of IE9 PP4.

Let’s look at Sunspider:

I have left out IE8 and its result of 4989.4 ms. The sluggish performance is widely know and somewhat irrelevant in this chart as it distorts the differences between the other browsers. What is truly remarkable is that Chrome 6 is close to be breaking through the 300 ms barrier. Chrome 5 was, by the way, the first browser that broke through 400 ms just about five months ago. The Chrome nightly builds (NB) are by far the fastest JavaScript browsers available right now, hands down. The first IE9 PP was faster than Firefox and IE9 has kept that lead until today. It is now substantially faster, but the distance to Firefox is about the same as it was in March of this year. The interesting part, however, is Mozilla’s current JM performance claim of 525 ms. Mozilla is shaving off milliseconds now virtually every day and we know that Mozilla wants to end up within 20% of Chrome 5 performance, which is somewhere around 350 ms these days. If Mozilla achieves its goal, it will end up just north of 400 ms and possibly in the range or slightly ahead of IE9 Beta. 

Microsoft has to be careful that it does not have to eat its own words and may see a déjà vu with IE9: Back in March 2008, it expected to be able to surpass Firefox 3’s performance only to see Mozilla pull ahead with its TM engine. Clearly, Firefox is not a done deal for IE9 yet. IE9 Beta has to deliver a stunning upgrade if it wants to beat Firefox’ JM engine.

So, what about HTML 5?

If you have been reading recent benchmark stories that were pitched by Microsoft, then you know that IE9 is the best HTML 5 browser around, thanks to its added HTML 5 support and  hardware accelerated rendering engine. You can actually check Microsoft’s graphics performance page and it is clear that IE9 is much better in these benchmarks than any other browser right now. Sorta … maybe ... maybe not …. ummm, no.

While Microsoft hardware (GPU) acceleration of certain content is truly stunning as multithreading web content in current browsers has not been possible so far, I have two main issues with Microsoft’s claims. First, Firefox is, on average, at least as good and occasionally better than IE9 in those benchmarks. Microsoft just did not activate the still developed hardware acceleration that has been available in Firefox since version 3.7a5. If you activate the feature, Firefox 4.0 breezes through most of Microsoft’s benchmarks just as fast as IE9 PP4.

Of course, Chrome, Safari and Opera do not support hardware acceleration yet. But I am just not sure how important this feature is right now. Seriously, how many HTML 5 web sites do you know that benefit from hardware acceleration? Sure, such websites will come, but we know that Google is working on hardware acceleration as well, Opera will have it in the not too distant future and I am sure Apple will have it as well, too, especially since Steve Jobs is so focused on killing Flash. The fact, however, is: This big IE9 advantage isn’t such a big advantage yet.

As for HTML 5 support, IE9 has still substantial gaps. PP4, for example, does not support the critical canvas element yet. According to Caniuse.com, IE9 only supports 81% of all HTML 5 requirements, while Chrome is at 88%, Firefox at 96%, Safari at 88% and Opera at 77%. In the html5test.com test run, IE9 scores 85 of 300 points. Chrome 6 Beta hits 227 points, Opera 10.61 166 points, Safari 5.0.1 214 points and Firefox 4 Beta 4-pre 199 points. IE9 may not be the best HTML 5 browser around – and the scenario is somewhat reminiscent of the introduction of IE8, when Microsoft said it would fully support web standards. Of course, we know that this was not exactly the case, as shown, for example, by the Acid 3 test: IE8 scored just 21/100 points. IE9 scores 83/100 in Acid 3, while Firefox is at 97/100 and all other major browsers at 100/100.

Conclusion

There is no doubt that IE9 will be a solid upgrade for loyal IE8 users, but not for Firefox or Chrome users. IE9 seems to be more nimble, but it isn’t quite as fast as it needs to be if it wants to play with the big boys. If the leaked screenshots in fact show the look and feel of the new IE9, then it will look old right out of the gate. Opera, Chrome, Safari and Firefox offer sleek new interfaces that represent a changed user behavior, while IE9 may look just like IE8 or IE7. It is a bit too early to draw a final conclusion already now, but there is a good chance that IE9 will not deliver on the expectation it has created. Any JavaScript performance below Firefox and the lack of HTML 5 support will leave Microsoft with a graphics hardware acceleration feature that is nice to haven but rather useless today – like a 600 hp super car that is mainly driven in commuter traffic.

Better isn’t good enough for IE9.

Discuss
Ask a Category Expert

Create a new thread in the News comments forum about this subject

Example: Notebook, Android, SSD hard drive

This thread is closed for comments
Top Comments
  • 26 Hide
    lejay , August 18, 2010 12:04 AM
    So the leaked screenshots of a unfinished product looks half-baked? Wow, what a surprise.
  • 20 Hide
    falchard , August 18, 2010 1:48 AM
    OMG a Google Benchmark on JavaScript shows IE9 is not good at it while Chrome is simply awesome. I never saw that one coming.
  • 18 Hide
    cookoy , August 18, 2010 12:25 AM
    anyone benchmark which browser runs tomshardware site fastest?
Other Comments
    Display all 63 comments.
  • 26 Hide
    lejay , August 18, 2010 12:04 AM
    So the leaked screenshots of a unfinished product looks half-baked? Wow, what a surprise.
  • 3 Hide
    IM0001 , August 18, 2010 12:05 AM
    It will definitely be interesting to see. From what Microsoft has learned over the last year is that they need to get things right the first time when releasing a new product (We all know about the Kin and I believe Microsoft learned from it big time). Microsoft really has a tough market to punch through here and with all hope going into it, the Beta (and more importantly, the final product) will deliver.
  • -4 Hide
    proxy711 , August 18, 2010 12:08 AM
    "I hate to crash Microsoft IE9 Beta party early, but it appears that the browser will miss its mark."

    "The performance we have seen so far is mediocre at best and the screenshots that were leaked make IE9 look like a half-baked product."

    Ya no surprise there. I hope no one was expecting anything other than the typical half baked product that IE is.
  • 3 Hide
    jimmysmitty , August 18, 2010 12:16 AM
    I have been messing with both IE9 PR4 and FF 4.0 PR2. Both are great. But FF looks a bit bland. I hate how they seem to be moving away from the standard bar to something less.

    And so far, IE9 has yet to not be impressive with the test results. You can easily go to either website that runs these so called browser tests and run them easily to compare your own results with theirs, minus anything related to MS since its based also on internet connection speed.

    Also, why in the hell would we use anything made by a company in the competition? Since when is it ok to use a benchmark like Googles? Do you know what it truly means? That Google spent tons of time tweaking Chrome to look amazing in that test.

    Chrome is meh. Google has a ok browser. Safari is a no go. Intrusive APple software, hah. Haven't had a need for Opera since FF and IE work great together.
  • 18 Hide
    cookoy , August 18, 2010 12:25 AM
    anyone benchmark which browser runs tomshardware site fastest?
  • 5 Hide
    willgart , August 18, 2010 12:36 AM
    Wait and see... comparing a PP version to existing and public beta versions is not the same...
    between PP1 and PP4 there is a huge improvement in all parts.
    and based on other "microsoft" benchmarks (like the google one) IE9 outperform any other browser in real life usage. which is opening pages and surfing pages, and not just doing benchmarks.

    but its not the final version... so... we wait...
  • 0 Hide
    xophaser , August 18, 2010 12:43 AM
    my firefox is starting to slow down. When I switched from ie 8 to firefox it was the fastest ever. Now with all the extensions it is lagging a little, and chrome, which I use also, seem faster. IE9 needs to be as fast/fastest and have the slick look of firefox 4 beta.
  • 3 Hide
    scione , August 18, 2010 12:57 AM
    no love for Opera?
  • 0 Hide
    jimmysmitty , August 18, 2010 1:00 AM
    willgartWait and see... comparing a PP version to existing and public beta versions is not the same...between PP1 and PP4 there is a huge improvement in all parts.and based on other "microsoft" benchmarks (like the google one) IE9 outperform any other browser in real life usage. which is opening pages and surfing pages, and not just doing benchmarks.but its not the final version... so... we wait...


    Pretty much. I have both IE9 PP4 and FF 4.0 Beta 2. So far, even with hardware acceleration IE9 PP4 is beating FF 4.0 Beta 2 in the test MS provides. They are tied for me in terms of speed in milliseconds since I have a T1 (1.5Mbps..... yay) at work.

    I for one am hopeful for the beta. It seems people don't realize that if IE9 does good, that means the other browsers try to do better which is a win win. Each and every one of them see MS IE as their main competition and they want to beat IE. If IE beats them, they will work harder to be better.

    Now when I look at IE(, I have hope too since so far Windows 7 is great, MS Security Essentials is great and have yet to have any problems with anything else.
  • 20 Hide
    falchard , August 18, 2010 1:48 AM
    OMG a Google Benchmark on JavaScript shows IE9 is not good at it while Chrome is simply awesome. I never saw that one coming.
  • -9 Hide
    tektek2000 , August 18, 2010 1:56 AM
    Good article.. you sir .. is TOM'S last hope as what tomshardware used to be!!

  • -8 Hide
    dEAne , August 18, 2010 2:24 AM
    yeah better is not enough for IE9...it needs to do more.
  • 2 Hide
    maddad , August 18, 2010 2:44 AM
    What I want to know doesn't really have anything to do with benchmarks per se. In everyday use, how much of a difference will you actually notice. Firefox is still my preferred browswer and runs well on my cheapo home built quad system. I have used google chrome and Opera, but I still go back to Firefox as my favorite. I rarely ever use IE8, but still have it on the computer for those Microsoft tasks that require it. The fact that Google is able to throw unlimited money and software engineers at Chrome to make it faster than Firefox does nothing to make me want to switch. Frankly, everything Google is no better than everything Microsoft. When the Google fat cat is the only, browser, operating system and search engine left will they still put out weekly updates? Or will it just be another bloated monster like Microsoft became.
  • 13 Hide
    danielgr , August 18, 2010 2:57 AM
    Are you sure you have run your benchmarks with the last version of IE9?
    You say you are using PP4 but everything in your article seems to indicate you used PP3. Appart from your obvious pro-google bias (shown all through your article quotes and by the fact that you provide a Google bench as a comparison tool), you should at least get your facts right. For PP4:
    - Acid score is 95%
    - HTML5test score is 96
    - At least on my computer i5-760 + HD4770 Spider runs faster than in Safari 5.0.1 (contrary to PP3).
    - Since PP3 IE9 supports HTML5 canvas element (and it's GPU accelerated, providing impressive and unmatched speed for something you qualified as "critical").
  • 7 Hide
    ethaniel , August 18, 2010 2:58 AM
    So what? A "preview" version is slower than a finished product. Hellooooo! IE9 doesn't even have a beta yet. Hell, there's not even an official date for the final release. The entire article is "almost trollish", and I think you'll have to wait a while for that...
  • -5 Hide
    randomizer , August 18, 2010 3:03 AM
    falchardOMG a Google Benchmark on JavaScript shows IE9 is not good at it while Chrome is simply awesome. I never saw that one coming.

    Then you must be living in a cave. IE and Firefox both have terribly slow JavaScipt engines compared to the others, that's just how it is. Fortunately this doesn't make a great deal of difference in most real-world uses, but some sites make very heavy use of JavaScript and that's where you'll see the two big market share holders crawling.

    That said, the only significant difference between Chrome 2.0 and Chrome 6.0 is performance. They're going through version numbers like they've just learned how to count, yet there's no significant changes. FF for me :) 
  • 0 Hide
    randomizer , August 18, 2010 3:06 AM
    ethanielSo what? A "preview" version is slower than a finished product. Hellooooo! IE9 doesn't even have a beta yet. Hell, there's not even an official date for the final release. The entire article is "almost trollish", and I think you'll have to wait a while for that...

    Chrome 6 is pre-release.
  • -8 Hide
    belardo , August 18, 2010 4:01 AM
    Hope IE9 crashes and burns... need to take more market share out of MS.

    I don't test with IE anymore, its has too many problems that don't show up on FF, Opera or Chrome. Dont like Safari enough to bother with hit.
  • -7 Hide
    number13 , August 18, 2010 4:04 AM
    OH Well
  • 0 Hide
    mikewong , August 18, 2010 4:16 AM
    I'm currently using Firefox and am happy with it, especially with its continuous updates.
Display more comments