Samsung Accused of Manipulating Benchmarks Again

[exfileme]

Ars Technica presents its evidence against Samsung.

Samsung Accused of Manipulating Benchmarks Again : Read more

 

[ohim]

I`m confused, so... technically Samsung is "cheating" because they allow their CPU to run at full speed sometimes? ... that`s a "shocker". And i still don`t get it why would that CPU idle 3 cores when running a benchmark .. isn`t that application suppose to max out the hardware?

 

[Disingenuous]

Samsung is "cheating" because they purposefully created a bit a code that checks for the identity of the app that is running. That bit of code has a hard coded list of names that it checks to see if the app that is running is a popular benchmarking app. When it detects that it is, it behaves differently by going full throttle on its cpu core speeds to affect the outcome of the benchmark scores even before the benchmark is running.

This is disingenuous since the performance seen on the benchmark is not truly what you can expect in real-world situations. Instead, the score has been overinflated thanks to that bit of code detecting the benchmarking app, thereby making people think that the device is more powerful than it truly is in real-world situations.

 

[qlum]

In short next time you benchmark phones make sure to modify the benchmark name first to get a fair score.

 

[greghome]

Well then.......Intel is cheating all the time with turboboost and Hyperthreading

 

[heero yuy]

so "benchmark mode" is where the processor runs at maximum and shows what the phone can do when on full belt
and that's cheating? I thought a benchmark is to see what the device can do when pushed

 

[laseru]

If you don't understand the basics better not write anything here. First it cheats because the phone in real life does not work like that. I hope you really understand that, it does not work like that in real life. If they do that at least they should say, but I am very curious why they don't do that with a game???? Because the battery which is very bad will be even worst maybe? Second the SoC on this phone the Snapdragon 800 has the same features as Turbo Boost, HyperThreading is not available here! Except that with Intel they don't need to keep all the cores online and at full speed when the pc is doing nothing just rendering the interface of a benchmark app. They should copy Intel and allow this boost of frequency on the fly but it seems for whatever reason if they do that the values are lower so that's the reason why they turn the CPU on and keep the cores running at 2,3Ghz doing nothing. Either way a cheap brand, cheap products(quality wise), cheap customization and add the regional lock to that and a high price. But there are people stupid enough to call this a premium device instead of a big scream device with average performance and a cartoonish OS. Don't get me wrong I don't hate Android as I own both IOS and Android but some things need t be said the right way.

 

[Muhibah]

Thats the keyword, all the time. Its ok to turbobooat to any real application all the tine but it is cheating if u only turbo boost on benchmark for the sake of showing off

 

[Muhibah]

Thats the keyword, all the time. Its ok to turboboost to any real application all the time as u benefit from it but it is cheating if u only turbo boost on benchmark for the sake of showing off

 

[house70]

Idiotic move on Samsung's part. The phone is plenty fast without crap like this.
Esp. since benchmarks now matter to only the few that still place any value into them (I am referring to phones benchmarks). For daily use they're totally irrelevant, people could give a rat's a$$ about bench results, and all is running the way it's supposed to run. Why risk getting caught like this, when you know the OS is open and can (and will) be examined by everybody with a dog in it?

 

[teh_chem]

Thats the keyword, all the time. Its ok to turbobooat to any real application all the tine but it is cheating if u only turbo boost on benchmark for the sake of showing off

I'm not defending what's been done, but that's not the case here. Turbo is generally available to any application, and is a low-level CPU load and TDP-dependent operating; not a software/code-dependent thing that only allows turbo under very specific conditions and applications. Manipulating CPU response in the way that Samsung is allegedly doing just makes benchmark comparisons fairly worthless--even more than they already were for extracting real-world usage experience.

 

[Lord Captivus]

What do you think the reason are for that, besides the higher score?
Is it a temp thing? Battery consumption? Im talking about the reason they dont use all cores 24/7.

 

[wemakeourfuture]

Shocked at the sheer stupidity of people who don't see what's wrong with this or support Samsung. And I thought Apple fanboys where bad!

 

[cats_Paw]

Well... we still have to wait a good amount of time till smartphones actually do things that are nice (like playing crysis), so I guess a few bench here or there aint gonna make much of a difference unless the OS is programed in a similar way.

 

[damric]

Next we will have people complaining about me setting my pc bios to my maximum overclock when running 3dmark, because I don't run my OC like that 24/7?

 

[haswell5271]

Do people really care that Samsung gave the benchmark score for the speeds their processors are capable of? Can't any app use full processing power? Am I wrong? I think Samsung can use any clock rate that they sell the phone with. Overclocking and THEN bench marking would be cheating IMHO.

 

[calam1ty]

I really don't understand how people think this isn't cheating... It's as if you have this great car that unlocks all of its horsepower and performance when a car reviewer decides to drive it, but then dumbs everything down to prius levels of drivability when anyone else drives it.

Sure this phone certainly has the potential to run great with all 4 cores maxed out but just like the article stated, what's the point if it'll never run like that for any other piece of software you decide to use that isn't a benchmark?

 

[damric]

Isn't it common sense to disable power saving features when using synthetic benchmarking?

 

[damric]

Isn't it common sense to disable power saving features when using synthetic benchmarking?

 

[stevejnb]

Isn't it common sense to disable power saving features when using synthetic benchmarking?

Are other phone companies doing it when they benchmark their phones? If not, then it's only "common sense" in that you're painfully aware you're using a different and advantageous set of rules compared to your competitors and trying to pawn it off as if you're playing by the same rules. Sounds like cheating to me.

I can understand this practice for some hypothetical "how powerful COULD this phone be"... But when you buy a phone that says "This phone is X powerful" but it will never be more than 80% of X when you actually use it because of some benchmarking trick like this, that's disingenuous.

It's not much different than the Surface RT having a 32gb SSD. Yes, in theory, absolutely true. When 14 or so gb of that is taken up prior to you even turning the device on for the first time, advertising 32gb of storage without a notable footnote is pretty disingenuous. Go figure, people were *flipping out* when MS misrepresented the storage space on their tablets like that...

 

[Bloob]

In short next time you benchmark phones make sure to modify the benchmark name first to get a fair score.

Next time you test battery life, make sure to use these benchmarks to test it.

 

[bombebomb]

I'm not sure how the note 3 run's, but with my ROOTED tf101 I can force it to stay maxed. Now, this phone is not rooted and it's "out of the box testing" so that point might be a little moot. However I must agree with some other folks, were bench marking the CPU, not the "phone". If the tests were done differently say, all cores hit 2.6ghz during benchmarks and nothing else that would throw lots of red flags. This is not the case though and I believe it's being way over hyped.

Guy 1: Yo, I benchmarked my intel core I7, it did a score of over 9000
Guy 2: Yeah, but you used the processor full potential, so it don't count cause it does not run at 5ghz when you click the start menu.

 

[bombebomb]

I'm not sure how the note 3 run's, but with my ROOTED tf101 I can force it to stay maxed. Now, this phone is not rooted and it's "out of the box testing" so that point might be a little moot. However I must agree with some other folks, were bench marking the CPU, not the "phone". If the tests were done differently say, all cores hit 2.6ghz during benchmarks and nothing else that would throw lots of red flags. This is not the case though and I believe it's being way over hyped.

Guy 1: Yo, I benchmarked my intel core I7, it did a score of over 9000
Guy 2: Yeah, but you used the processor full potential, so it don't count cause it does not run at 5ghz when you click the start menu.

 

[warezme]

I see it as a tool to mod and add my own apps onto this list and make those apps run faster. I'll probably need a charger handy though.

 

[kristi_metal]

This is because people care too much about the synthetic benchmarks, and they shouldn't, those benchies mean nothing to real-life usage