Sign in with
Sign up | Sign in

Samsung Accused of Inflating Galaxy S4 Benchmarks

By - Source: AnandTech | B 31 comments

Samsung responds to accusations of inflating benchmarks for the Galaxy S 4.

There are several reports now claiming that Samsung has cranked up the GPU clock speed in two international Galaxy S 4 smartphones to 533 MHz in certain benchmarks while keeping the GPU at 480 MHz in other benchmarks of apps and games. The reports also point to a string within an app for the GS4 that reveals something called "BenchmarkBooster" which overclocks the SoCs when specific benchmark apps are running.

For starters, the GPU frequency can be easily obtained over adb: 480 MHz. But when tested in GLBenchmark 2.5.1, the GPU clock speed increases up to 532 MHz as it also does in AnTuTu and Quadrant. However GFXBench 2.7.0 (formerly GLBenchmark 2.7.0) shows the 480 MHz clock rate. There's speculation that this latter benchmark isn't allowed, aka white-listed, to run the GPU at the higher setting.

The performance boost isn't locked to the GPU portion of Samsung's Exynos 5 Octa chip either. "Using System Monitor I kept an eye on CPU frequency while running the same tests," reports AnandTech. "Firing up GLBenchmark 2.5.1 causes a switch to the ARM Cortex A15 cluster, with a default frequency of 1.2 GHz. The CPU clocks never drop below that, even when just sitting idle at the menu screen of the benchmark."

But when running GFXBench 2.7, the chip switches over to the Cortex-A7 cluster running at 500 MHz (250 MHz virtual frequency). Thus it would appear that GLBenchmark 2.5.1 is the only benchmark allowed to run the higher performance mode. AnTuTu, Linpack, Benchmark Pi, and Quadrant reveals the same behavior: a fixed CPU governor.

"Note that the CPU behavior is different from what we saw on the GPU side however," the AnandTech report states. "These CPU frequencies are available for all apps to use, they are simply forced to maximum (and in the case of Snapdragon, all cores are plugged in) in the case of these benchmarks. The 532MHz max GPU frequency on the other hand is only available to these specific benchmarks."

As for "BenchmarkBooster", it's a string within the TwDVFSApp.apk that's actually allowing the frequency changes. This string specifically names benchmarks that are allowed to use the higher settings including Quadrant standard/advanced/professional, Linpack (free, not paid), Benchmark Pi, and AnTuTu.

Naturally once the report surfaced and began to spread, Samsung came forth to defend its benchmarks:

Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

Samsung Electronics remains committed to providing our customers with the best possible user experience.

Discuss
Ask a Category Expert

Create a new thread in the News comments forum about this subject

Example: Notebook, Android, SSD hard drive

This thread is closed for comments
  • -5 Hide
    m32 , August 1, 2013 3:34 AM
    So? I don't really see the problem here. If they do this for other tasks (video playback & etc) and their chip can do it, why not? I think mobile devices are way too dependent on benchmarks anyway. No one seems to do the eye test anymore.
  • 8 Hide
    srap , August 1, 2013 4:05 AM
    Quote:
    So? I don't really see the problem here. If they do this for other tasks (video playback & etc) and their chip can do it, why not?


    I would like to highlight this part for you:
    Quote:
    The 532MHz max GPU frequency on the other hand is only available to these specific benchmarks.
  • -8 Hide
    house70 , August 1, 2013 4:58 AM
    Quote:

    I would like to highlight this part for you:
    Quote:
    The 532MHz max GPU frequency on the other hand is only available to these specific benchmarks.


    I would like to highlight this part for you:
    "Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance."

    Anyhow, a benchmark is supposed to push the hardware to its limit. If the limit is higher than what your hardware usually uses, that's only because certain programs don't push it to the limit, like a benchmark does.

    I never check benchmark results when buying phones. Never. All I care about are battery size and some minimum specs.
  • Display all 31 comments.
  • 2 Hide
    pacomac , August 1, 2013 5:03 AM
    The S4 has never been a slouch in terms of CPU benchmarks, but the iPad was giving it a run for it's money in OpenGL benchmarks. Samsung obviously felt threatened otherwise they wouldn't be cheating in this way. Why would they need a list of benchmark programs listed in the apk unless they were making exceptions for these programs?
  • 0 Hide
    sundragon , August 1, 2013 5:13 AM
    1. Way to go Samsung, you make great hardware and you don't need to inflate your benchmarks.

    2. Apple still spanks your GPU when it comes to same generation hardware - make better GPUs, it's not rocket science...

    3. Using Franco Kernel, I can dynamically push my Nexus 7's GPU for specific games - It would be great if Samsung gave the users that ability but doing it behind your back and then only for specific apps (benchmarks) speaks volumes.

    4. I would love to see the same people make the same excuses if Apple had done the same... Inflating benchmark results beyond what normal apps use deceitful.
  • 0 Hide
    classzero , August 1, 2013 5:14 AM
    Busted!
  • 5 Hide
    wemakeourfuture , August 1, 2013 5:32 AM
    Quote:
    house70
    Anyhow, a benchmark is supposed to push the hardware to its limit. If the limit is higher than what your hardware usually uses, that's only because certain programs don't push it to the limit, like a benchmark does.

    I never check benchmark results when buying phones. Never. All I care about are battery size and some minimum specs.


    Do you understand what they're doing? They're essentially putting explicit instructions that when a benchmark is being done, inflate its GPU clock speed. Benchmarks are to get accurate results of its performance, Samsung is trying to fool it to give misleading results so its product has more marketing value.

    A benchmark pushes the device to get its upper-bounds not be tricked into getting results that non-benchmark apps would get.

    Whether you use the specs in your purchasing is one thing, but trying to fool results so stats are inflated for the sake of marketing is wrong in me eyes.

    If Apple did this you it would have a 100 comments by now...
  • 1 Hide
    cmi86 , August 1, 2013 5:33 AM
    Oh well it's a lot less scandalous than the crap apple pulls almost on a weekly basis..
  • -1 Hide
    halcyon , August 1, 2013 5:36 AM
    Sammy doesn't need to do this. The S4 is awesome and most already know that. It's just a phone anywho, why so serious? Besides, I'm sure Samsung is not the only one doing this.
  • -4 Hide
    eiskrystal , August 1, 2013 6:22 AM
    Unless i'm missing something i'm not seeing how it's cheating to show your full power on a test of your full power.

    The Tegra 3 can switch on different cores depending on workload. Is that also cheating if all 4 cores fire up during a heavy test?
  • -3 Hide
    eiskrystal , August 1, 2013 6:23 AM
    Unless i'm missing something i'm not seeing how it's cheating to show your full power on a test of your full power.

    The Tegra 3 can switch on different cores depending on workload. Is that also cheating if all 4 cores fire up during a heavy test?
  • -5 Hide
    eiskrystal , August 1, 2013 6:23 AM
    Unless i'm missing something i'm not seeing how it's cheating to show your full power on a test of your full power.

    The Tegra 3 can switch on different cores depending on workload. Is that also cheating if all 4 cores fire up during a heavy test?
  • 0 Hide
    eiskrystal , August 1, 2013 6:26 AM
    Given how awkward and buggy reading and making comments is, I can only assume Toms doesn't want us to do it.
  • 1 Hide
    sundragon , August 1, 2013 6:33 AM
    Quote:
    Oh well it's a lot less scandalous than the crap apple pulls almost on a weekly basis..


    Wow dude, you should wipe your mouth and nose a bit more before being a Samsung apologist...
  • -2 Hide
    rwinches , August 1, 2013 6:43 AM
    Tricky wordsmithing by AnandTech.
    Tom's will often run performance tests that make sure all cores run at their max.
    480 normal max 533 turbo max big deal.
    How many benchmarks and programs used in reviews are geared for Intel?
    A phone purchase is based on features like the cool enhanced OLED displays in the S4
  • 7 Hide
    sundragon , August 1, 2013 6:48 AM
    Quote:
    Tricky wordsmithing by AnandTech.
    Tom's will often run performance tests that make sure all cores run at their max.
    480 normal max 533 turbo max big deal.
    How many benchmarks and programs used in reviews are geared for Intel?
    A phone purchase is based on features like the cool enhanced OLED displays in the S4


    Deceiving your customers is not acceptable (Period)

    I don't understand how people can apologize for a company that actively tries to be deceitful? Again, if this were Apple, you'd be all up in arms. Double standards, much?

    And every test geared for Intel gets called out! There was recently one where Intel made the benchmark skew the new Atom's performance/power to look better than ARM A15. They got called out on it.
  • -1 Hide
    wh3resmycar , August 1, 2013 6:49 AM
    benchmarks don't matter in a smartphone.... as if you'd be able to switch graphics.. buy the damn phone for its features
  • 0 Hide
    dejavu117 , August 1, 2013 7:28 AM
    Silly Samsung, Tricks are for kids!
  • 2 Hide
    wemakeourfuture , August 1, 2013 7:33 AM
    Quote:
    wh3resmycar
    benchmarks don't matter in a smartphone.... as if you'd be able to switch graphics.. buy the damn phone for its features


    I agree for most customers, so why would Samsung go out of its way to fool benchmarks to give higher results than expected by the apps the end users would use?
  • 2 Hide
    wemakeourfuture , August 1, 2013 7:39 AM
    Quote:
    eiskrystal

    Unless i'm missing something i'm not seeing how it's cheating to show your full power on a test of your full power.

    The Tegra 3 can switch on different cores depending on workload. Is that also cheating if all 4 cores fire up during a heavy test?


    You are missing something. There's a difference between having architecture that boosts based on resource intensive cases compared to having explicit instructions to only produce higher results when benchmarking software is being used.

    There's CPU's that boost the clock speed when needed, this is an architecture design to improve performance. Then you have Samsung that says lets boost clock speed when a benchmark is being performed so results are inflated and not a true representation of the hardware performance.
Display more comments