Vista: Benchmarking or Benchmarketing?

Windows Marketplace: Search For Suitable Software

Microsoft introduced the Windows Marketplace, where users can look for software based on the Windows Experience Index. This website only displays software titles that will run on your Windows Vista system.

If you take a close look at the hyperlink above, you will realize that it carries parameters for all five sub-indices: CPU, MEM, HDD, DWM and D3D. You will find such a link at Control Panel\System and Maintenance\Performance Information and Tools ("View software for my base score online"), but you can manually exchange the performance indices for each component if you want to.

While the intention of the Windows Marketplace certainly is nice, you should not forget that this is a Microsoft offering, which does not display any open source software: the offer is purely commercial.

Conclusion

Microsoft's Windows Experience Index is a benchmarking solution built into Windows Vista that can be used to estimate whether a hardware component will be powerful enough to run particular software, or if a software product you intend to buy will run on your system. We definitely recommend spending some time on the WEI, as we expect it to appear on more and more retail products. Products that rate at 5 or higher can definitely be called upper mainstream or high-end, though we recommend against trusting only the WEI. However, the index is intentionally called the Windows Experience Index, not the computer or component experience index. It reflects components' ability to run Windows Vista and their support for the latest features, but it is by no means a replacement for traditional benchmarking.

We found substantial differences when comparing the Windows Experience Indices for different processors and graphics cards. If you only looked at the Windows Experience Index, you could end up purchasing a product which will not satisfy your requirements! Vista rated all of our graphics cards at high scores of 4.5 and up, meaning that they supposedly are suitable to run 3D game graphics, but as our game and 3DMark results show, this is absolutely wrong.

The same applies to processors: depending on your application, a dual or quad core processor can effectively double or quadruple performance. The index rates state-of-the-art processors at level 5, while even a three year old processor received a rating level of 4. It may be powerful enough to run Windows, but as soon as you have application requirements, the only way to find an appropriate product is to compare more benchmark results.

Clearly, the Windows Experience Index is welcome, because it helps provide a general impression on of component and system performance and capabilities. It cannot, however, replace traditional benchmarking, due to its limited benchmarking horizon.

Join our discussion on this topic