Samsung Accused of Manipulating Benchmarks Again

Samsung is again accused of manipulating benchmarks, this time so that the Galaxy Note 3 scores really well. The company was previously accused of inflating Galaxy S4 benchmarks earlier this summer, as the GPU and CPU clock speeds would change depending on the benchmark used. Now Ars Technica has discovered another spec inflation with this latest Note 3 phone.

The site claims that the 2.3 GHz Snapdragon 800 chip used in Samsung's Galaxy Note 3 phone highly outperforms the same chip used in the similarly specced LG G2 phone. After a little investigating, the site discovered that Samsung is supposedly artificially boosting the U.S. Note 3's scores with a special high-power CPU mode that kicks in when a large number of popular benchmarks are running.

According to the report, when the Note 3 is idling normally, three of the four cores are turned off while the fourth is idled down to 300 MHz. However when running a benchmark app, all four cores are forced to remain active and maxed out at 2.3 GHz – they are not allowed to shut off. Even though idling has nothing to do with benchmarking scores, this was the first sign of foul play, as a device shouldn't treat a benchmarking app differently than a normal app.

Ars Technica then discovered that the phone goes into benchmarking mode when the CPU detects specific benchmarking apps. For instance, the site disassembled the Geekbench 3 app, renamed the package as Stealthbench 3, and then reassembled it. The Note 3 went into benchmark mode when Geekbench was loaded, but not when the renamed app was loaded.

Geekbench also showed that in System Monitor, all four cores were active and maxed out at 2.3 GHz, whereas the renamed app revealed that the same chip turned off three cores and idled down the fourth to 300 MHz. Additional testing revealed that the Note 3's benchmarking mode gives the device a 20 percent boost over its natural score, which is similar to the LG G2 phone, if not slightly better.

The benchmark mode is triggered by a file named "DVFSHelper.java" that contains a hard-coded list of every package that is affected by the CPU boosting. The "DVFS" in "DVFSHelper" actually stands for "Dynamic frequency scaling", also known as CPU throttling, the site reports. This DVFSHelper function is used exclusively for benchmarks, Ars claims, covering all the popular benchmarking apps including Geekbench, Quadrant, Antutu, Linpack, GFXBench and several Samsung benchmarks.

Ars Technica continues to plead its case against Samsung here in this report.

Create a new thread in the US News comments forum about this subject
This thread is closed for comments
71 comments
    Your comment
    Top Comments
  • calam1ty
    I really don't understand how people think this isn't cheating... It's as if you have this great car that unlocks all of its horsepower and performance when a car reviewer decides to drive it, but then dumbs everything down to prius levels of drivability when anyone else drives it.

    Sure this phone certainly has the potential to run great with all 4 cores maxed out but just like the article stated, what's the point if it'll never run like that for any other piece of software you decide to use that isn't a benchmark?
    14
  • Disingenuous
    Samsung is "cheating" because they purposefully created a bit a code that checks for the identity of the app that is running. That bit of code has a hard coded list of names that it checks to see if the app that is running is a popular benchmarking app. When it detects that it is, it behaves differently by going full throttle on its cpu core speeds to affect the outcome of the benchmark scores even before the benchmark is running.

    This is disingenuous since the performance seen on the benchmark is not truly what you can expect in real-world situations. Instead, the score has been overinflated thanks to that bit of code detecting the benchmarking app, thereby making people think that the device is more powerful than it truly is in real-world situations.
    13
  • Other Comments
  • ohim
    I`m confused, so... technically Samsung is "cheating" because they allow their CPU to run at full speed sometimes? ... that`s a "shocker". And i still don`t get it why would that CPU idle 3 cores when running a benchmark .. isn`t that application suppose to max out the hardware?
    -6
  • Disingenuous
    Samsung is "cheating" because they purposefully created a bit a code that checks for the identity of the app that is running. That bit of code has a hard coded list of names that it checks to see if the app that is running is a popular benchmarking app. When it detects that it is, it behaves differently by going full throttle on its cpu core speeds to affect the outcome of the benchmark scores even before the benchmark is running.

    This is disingenuous since the performance seen on the benchmark is not truly what you can expect in real-world situations. Instead, the score has been overinflated thanks to that bit of code detecting the benchmarking app, thereby making people think that the device is more powerful than it truly is in real-world situations.
    13
  • qlum
    In short next time you benchmark phones make sure to modify the benchmark name first to get a fair score.
    9