Samsung Accused of Manipulating Benchmarks Again
Ars Technica presents its evidence against Samsung.
Samsung is again accused of manipulating benchmarks, this time so that the Galaxy Note 3 scores really well. The company was previously accused of inflating Galaxy S4 benchmarks earlier this summer, as the GPU and CPU clock speeds would change depending on the benchmark used. Now Ars Technica has discovered another spec inflation with this latest Note 3 phone.
The site claims that the 2.3 GHz Snapdragon 800 chip used in Samsung's Galaxy Note 3 phone highly outperforms the same chip used in the similarly specced LG G2 phone. After a little investigating, the site discovered that Samsung is supposedly artificially boosting the U.S. Note 3's scores with a special high-power CPU mode that kicks in when a large number of popular benchmarks are running.
According to the report, when the Note 3 is idling normally, three of the four cores are turned off while the fourth is idled down to 300 MHz. However when running a benchmark app, all four cores are forced to remain active and maxed out at 2.3 GHz – they are not allowed to shut off. Even though idling has nothing to do with benchmarking scores, this was the first sign of foul play, as a device shouldn't treat a benchmarking app differently than a normal app.
Ars Technica then discovered that the phone goes into benchmarking mode when the CPU detects specific benchmarking apps. For instance, the site disassembled the Geekbench 3 app, renamed the package as Stealthbench 3, and then reassembled it. The Note 3 went into benchmark mode when Geekbench was loaded, but not when the renamed app was loaded.
Geekbench also showed that in System Monitor, all four cores were active and maxed out at 2.3 GHz, whereas the renamed app revealed that the same chip turned off three cores and idled down the fourth to 300 MHz. Additional testing revealed that the Note 3's benchmarking mode gives the device a 20 percent boost over its natural score, which is similar to the LG G2 phone, if not slightly better.
The benchmark mode is triggered by a file named "DVFSHelper.java" that contains a hard-coded list of every package that is affected by the CPU boosting. The "DVFS" in "DVFSHelper" actually stands for "Dynamic frequency scaling", also known as CPU throttling, the site reports. This DVFSHelper function is used exclusively for benchmarks, Ars claims, covering all the popular benchmarking apps including Geekbench, Quadrant, Antutu, Linpack, GFXBench and several Samsung benchmarks.
Ars Technica continues to plead its case against Samsung here in this report.

Sure this phone certainly has the potential to run great with all 4 cores maxed out but just like the article stated, what's the point if it'll never run like that for any other piece of software you decide to use that isn't a benchmark?
This is disingenuous since the performance seen on the benchmark is not truly what you can expect in real-world situations. Instead, the score has been overinflated thanks to that bit of code detecting the benchmarking app, thereby making people think that the device is more powerful than it truly is in real-world situations.
This is disingenuous since the performance seen on the benchmark is not truly what you can expect in real-world situations. Instead, the score has been overinflated thanks to that bit of code detecting the benchmarking app, thereby making people think that the device is more powerful than it truly is in real-world situations.
and that's cheating? I thought a benchmark is to see what the device can do when pushed
Esp. since benchmarks now matter to only the few that still place any value into them (I am referring to phones benchmarks). For daily use they're totally irrelevant, people could give a rat's a$$ about bench results, and all is running the way it's supposed to run. Why risk getting caught like this, when you know the OS is open and can (and will) be examined by everybody with a dog in it?
I'm not defending what's been done, but that's not the case here. Turbo is generally available to any application, and is a low-level CPU load and TDP-dependent operating; not a software/code-dependent thing that only allows turbo under very specific conditions and applications. Manipulating CPU response in the way that Samsung is allegedly doing just makes benchmark comparisons fairly worthless--even more than they already were for extracting real-world usage experience.
Is it a temp thing? Battery consumption? Im talking about the reason they dont use all cores 24/7.
Sure this phone certainly has the potential to run great with all 4 cores maxed out but just like the article stated, what's the point if it'll never run like that for any other piece of software you decide to use that isn't a benchmark?
Are other phone companies doing it when they benchmark their phones? If not, then it's only "common sense" in that you're painfully aware you're using a different and advantageous set of rules compared to your competitors and trying to pawn it off as if you're playing by the same rules. Sounds like cheating to me.
I can understand this practice for some hypothetical "how powerful COULD this phone be"... But when you buy a phone that says "This phone is X powerful" but it will never be more than 80% of X when you actually use it because of some benchmarking trick like this, that's disingenuous.
It's not much different than the Surface RT having a 32gb SSD. Yes, in theory, absolutely true. When 14 or so gb of that is taken up prior to you even turning the device on for the first time, advertising 32gb of storage without a notable footnote is pretty disingenuous. Go figure, people were *flipping out* when MS misrepresented the storage space on their tablets like that...
Next time you test battery life, make sure to use these benchmarks to test it.