Update: Intel Expands Our Battery Testing At CES 2013

Why You Need To Look At Tablet Power Benchmarks Critically

At this year's CES, we had a chance to spend several hours with Intel engineers in the company's suite at the Venetian, breaking down the power consumption of four new tablets based on Intel's own Atom Z2760, Nvidia's Tegra 3, and Qualcomm's APQ8060. 

Benchmarking is tricky business. When testing graphics cards, we’re accustomed to presenting performance as an average frame rate over time. But we know that isn’t an absolute representation. When we evaluate storage, we run corner cases and real-world transfer tests. But, again, we know those metrics don’t necessarily characterize every aspect of how drives behave, particularly since drive behavior changes over time. Similarly, when measuring how much power a tablet consumes and how long its battery lasts under a given workload, keep one thing in mind: that measurement is really only good for one specific test, and therefore not necessarily reflective of what you plan to do with the device. The graphics and storage considerations are nothing new, but when it comes to power testing tablets, how much can the benchmark workload really affect the outcome?

Plenty, it turns out. Enough to change the finishing order in our battery testing.

Testing Power Granularly, Again

Why go to all of this trouble in the first place? Well, Intel is naturally concerned about the perception that its x86 ISA is more complex, and therefore less viable in the mobile space. The company is eager to dig in as deep as possible to as many devices as possible based on its own technology and the competition. Alan’s piece, ARM Vs. x86: The Secret Behind Intel Atom's Efficiency, covered much of that though, illustrating the power performance of both Atom and Tegra 3 using defined benchmarks.

We’re interested because the hardware has been, up until now, so black-box. You have a smartphone or a tablet, you run what limited tests there are available for a given mobile operating environment, and you try to draw comparisons to dissimilar architectures.

But what if the Web browsing and video playback battery life tests you’ve seen up until now weren’t telling you as much as they possibly could? What if connecting leads to the battery, processing cores, graphics engine, and panel shed even more light onto how the tablet you have your eye on might behave under the tasks you typically perform?

After running three Web-based workloads and a pair of video playback tests, it quickly became clear that drawing sweeping conclusions about power is uninformed at best, and misleading at worst.