Earlier this summer, Microsoft unveiled a few battery life tests that, perhaps unsurprisingly, showed its own Edge browser to be the winner. As the methodology wasn’t fully published, some companies, including Opera, weren’t too happy with Microsoft’s test and results. Microsoft is now open sourcing its lab test and methodology, so that even if the tests favor its browser, at least other browsers can now optimize for the same tests.
Winning At Its Own Tests
Back in June, Microsoft performed three tests to prove that its Edge browser is indeed the most power efficient of them all. In the first test, the company pitted Edge, Chrome, Firefox, and Opera against each other, each running the same web workloads on separate and identical devices. At the end of the test, Microsoft would simply check how much battery life each of the identical devices had left, and conclude which of the browsers was the most efficient.
In the second test, Microsoft checked its own Windows 10 telemetry statistics to see which browser is more efficient, and again Edge seems to have been the winner with the lowest average power consumption.
In the third test, the company put a video on a loop on each browser and let them run down the computers’ batteries. Edge, again, seems to have won by a significant margin, as the device (Surface Book) on which it was running lasted the longest.
Open Source Benchmark
The problem with a vendor’s own benchmark is that, for one, it may be optimizing its product for things that the competition isn’t, therefore easily making its product the “winner.” To be fair to Microsoft, battery efficiency is quite an important factor, and one on which all browsers and apps should focus more.
Second, even if other competitors want to optimize for the exact same things, they may not be able to do so because they have no idea of what exactly were the tests comprised. Even when testing for “battery efficiency,” there are all sorts of technical details that may make the difference between which browser is winning and which isn’t.
Microsoft is now releasing its lab test code (with a few changes from the original one) and publishing the full methodology it used in its own tests.
Even so, there are caveats. The other vendors may simply not want to optimize their browsers for the same things Microsoft is optimizing in Edge. This could be especially true if the other browsers would have to compromise in other areas, such as security, or their compatibility with other platforms.
Now that Microsoft’s test is open source, it may be a good idea for Microsoft and the other browser vendors to shake hands and try to create an effective universal browser battery benchmark that they can all agree is fair. Then they can all try to optimize their browsers against that benchmark and compete with each other in that way.
