Benchmark Cheaters Galaxy S4, Note 3 Absolved in 4.4 Update

Last fall, several mobile phone manufacturers were caught cheating in Android benchmarks. HTC, LG, Asus, and Samsung were all found to be boosting clock rates when the devices ran specific benchmarks. In response, Futuremark delisted offending devices from its 3DMark performance rankings.

"That decision won us lots of praise from 3DMark users, even from those who owned one of the delisted models," said Jani Joki, director of engineering at Futuremark.

In light of the latest Android 4.4 KitKat update, Futuremark informed Tom's Hardware that the Samsung Galaxy Note III and Galaxy S4 now conform to its benchmark guidelines.

"We knew that delisting would send a message to manufacturers too. We hoped it would encourage them to be fair and honest with their customers," Joki added. "I am happy to say that our actions have paid off."

Samsung Galaxy S4

• 3DMark scores are valid when using Android 4.2.2, 4.4.x or later*
• Delisted when using Android 4.3 or 4.3.1

Samsung Galaxy Note III

• 3DMark scores are valid when using Android 4.4.2 or later*
• Delisted when using Android 4.3

As a result, Galaxy S4 and Galaxy Note III devices are now back on 3DMark's device channel.

We ran some tests of our own to see how Samsung Galaxy S4 performance is affected when going from Jelly Bean to KitKat.

In terms of raw score, we're pleased to see that there are some real performance gains in Android 4.4 KitKat. While 4.3 Jelly Bean still leads the pack in overall score, those numbers are tainted by an artificially-boosted clock frequency. In fact, the scores from 4.4 KitKat show an improvement in graphics performance over both 4.2.2 and the cheating 4.3.

Our testing led us to discover another interesting Android 4.4 KitKat-related performance quirk: the Samsung Galaxy S4 ROM throttles the SoC much more aggressively than it did in 4.2 and 4.3, making 4.4 a completely differently performing animal.

Follow Marcus Yam @MarcusYam. Follow us @tomshardware, on Facebook and on Google+.