Jump to content

Samsung cheating in benchmarks? Say it isn't so!


bigsnake49

Recommended Posts

I think it's just superfluous to do that, and as the article stated it's optimizations that are needed. Still to this day my HTC ONE runs as if it were brand new. The S4 may have higher marks in benchmarks, but in day to day use it isn't as good as the ONE.

 

(IMO)

 

 

-Luis

Link to comment
Share on other sites

Go here for the gory details:

 

http://arstechnica.com/gadgets/2013/07/samsung-allegedly-boosting-benchmark-performance/

 

Can't say I'm surprised.

Not because I'm a Samsung fan boy.. Because I'm not.. Owned more than my share of HTC phones..

 

But if you go into an article to prove something you probably can given the correct apps to do so. It's not in Samsung or any other manufacturers interest to fluff bench mark tests(which are highly subjextive) because anyone can redo the tests. I'm sure if you look into the authors history he's probably got an alter to Steve Jobs that his phone is plugged into getting power from the other side..

 

My point is that given any number of tests you can create an outcome.. IPhone fanatics say it about Sammy. And can "prove" it.. And Sammy fans the same thing about apple... Or any other manufacturer..

 

Given the variables in s4 design I'm not surprised someone with a bent against them is out there making claims... Happens after every phone is released...

Link to comment
Share on other sites

Not because I'm a Samsung fan boy.. Because I'm not.. Owned more than my share of HTC phones..

 

But if you go into an article to prove something you probably can given the correct apps to do so. It's not in Samsung or any other manufacturers interest to fluff bench mark tests(which are highly subjextive) because anyone can redo the tests. I'm sure if you look into the authors history he's probably got an alter to Steve Jobs that his phone is plugged into getting power from the other side..

 

My point is that given any number of tests you can create an outcome.. IPhone fanatics say it about Sammy. And can "prove" it.. And Sammy fans the same thing about apple... Or any other manufacturer..

 

Given the variables in s4 design I'm not surprised someone with a bent against them is out there making claims... Happens after every phone is released...

 

 

Here is the latest and greatest from Anandtech:

 

http://www.anandtech.com/show/7192/update-on-gpu-optimizations-galaxy-s-4

 

Are you you also accusing Anand of being a Steve Jobs clone or wannabe? Really? That's your comeback? Read the article, grasshopper!

Link to comment
Share on other sites

It's not cheating if your hardware can do it. The only "cheat" is to detect a benchmarking app and lock the CPU and GPU at maximum speed, aka disable power management. That's not cheating, that's giving accurate results that the hardware can do at best.

 

Honestly, it seems silly to not allow the higher frequency for all apps, but it likely has such a horrific battery drain (or can damage the device) that they lock it for most.

 

Also, this is almost certainly something that a custom kernel and some root fun will be able to change... :D

 

And yes, the question is, is this behavior exhibited in the Snapdragon version? I highly doubt it, you're running an entirely different GPU on the SoC.

Link to comment
Share on other sites

Here is the latest and greatest from Anandtech:

 

http://www.anandtech.com/show/7192/update-on-gpu-optimizations-galaxy-s-4

 

Are you you also accusing Anand of being a Steve Jobs clone or wannabe? Really? That's your comeback? Read the article, grasshopper!

 

That wasn't my comeback.. That was a joke.. My "comeback" was that these stories come out every time a phone comes out.. If you look for negative results.. You will find them.. There are always claims about phones.. They used one variation of a phone.. And claims that they got different results. To be accurate they would need to test each phone and use the same benchmark apps that Samsung did.. On every variation of the phone. Not cherry pick looking for negativity..

 

Like I said before this crap comes out every time a phone is released you have to take them with a grain of salt. I guarantee there are people right now provingthese results are in error.. TThat's why you should use real life things when looking at a phone for you.. If you pick phones based on benchmark tests you will either be really happy or really annoyed depending on who's "results" you look at..

 

Perhaps these test are right on accurate.. But after so many of these over the years you learn not to pay attention to them.. Because no phone operates underbenchmark conditions.. Your experience with a pphone and someone else's can be completely different based on apps you use.. If you like widgets or not... Games and power apps..

 

That's the real test not a benchmark under certain conditions.. And given the variables of benchmarks themselves you can't get anyone to agree which tests are the best..

 

So tiny cricket... You can believe one test and ignore all others.. Like a true fanboy or you can be open to all answers and all tests and realize most of them are crap.. And the only true test is the one a person makes not reading an article to get "expert" opinions.

Link to comment
Share on other sites

That wasn't my comeback.. That was a joke.. My "comeback" was that these stories come out every time a phone comes out.. If you look for negative results.. You will find them.. There are always claims about phones.. They used one variation of a phone.. And claims that they got different results. To be accurate they would need to test each phone and use the same benchmark apps that Samsung did.. On every variation of the phone. Not cherry pick looking for negativity..

 

Like I said before this crap comes out every time a phone is released you have to take them with a grain of salt. I guarantee there are people right now provingthese results are in error.. TThat's why you should use real life things when looking at a phone for you.. If you pick phones based on benchmark tests you will either be really happy or really annoyed depending on who's "results" you look at..

 

Perhaps these test are right on accurate.. But after so many of these over the years you learn not to pay attention to them.. Because no phone operates underbenchmark conditions.. Your experience with a pphone and someone else's can be completely different based on apps you use.. If you like widgets or not... Games and power apps..

 

That's the real test not a benchmark under certain conditions.. And given the variables of benchmarks themselves you can't get anyone to agree which tests are the best..

 

So tiny cricket... You can believe one test and ignore all others.. Like a true fanboy or you can be open to all answers and all tests and realize most of them are crap.. And the only true test is the one a person makes not reading an article to get "expert" opinions.

 

You obviously don't visit the review sites every time benchmarks are run! Phone geeks get in a lather about the tiniest little improvement in a benchmark. And this "improvement" was more than 10% over what the phone would normally run at. It's one thing to say here are the benchmarks at the normal speed and here are the same benchmarks when you overclock the phone. BTW, make sure you have your air conditioner run at 60F and blowing directly on the phone, otherwise it will go up in smoke.

 

And I will condemn Apple if they did the same thing.

Link to comment
Share on other sites

I think 2 different topics are getting confused here.

1. The validity of benchmarks.

2. Samsung intentionally skewing the results.

 

I am of the camp that benchmarks do very little to show how a phone will perform in real life conditions. The only way to know if a phone is "fast" enough for YOU is to use the phone under YOUR normal usage patterns. You have 14 days to test drive the phone. Use them.

 

It does appear that Samsung went out of their way to make sure that the S4 performed very well under certain conditions.

 

Brian Klug is extremely knowledgeable about the hardware used in all of the most popular smartphones on the market. It is highly unlikely that he is reporting these findings with malice. There are very few people writing on mainstream (if anandtech is considered mainstream) tech sites that possess his level of knowledge on matters related to the latest processors. To claim he is biased is pretty silly.

 

Bottom line, Samsung knows that several of the most read review sites post benchmark comparisons with their reviews. Samsung also know that the majority of people base their opinion of a product based on 3 sources. Online review websites, friends and family, and customer reviews left on retail sites. They played the system to help their phone look better. Read a lot of the initial reviews for the S4, a large amount of them stated that the S4 was laggy and stuttered, but the benchmarks said it was lightning quick. Seems to be a little disconnect between the 2.

 

I am not sure if I think Samsung is "bad" for doing this. They are doing what is best for their product. Seems like a smart thing to do. As long as people continue to think benchmarks mean a damn thing, I expect manufacturers to do their best to make sure that their products look good when they are benchmarked.

 

Just my 2¢

 

Sent from my SPH-L710 using Tapatalk 4 Beta

 

 

  • Like 1
Link to comment
Share on other sites

You obviously don't visit the review sites every time benchmarks are run! Phone geeks get in a lather about the tiniest little improvement in a benchmark. And this "improvement" was more than 10% over what the phone would normally run at. It's one thing to say here are the benchmarks at the normal speed and here are the same benchmarks when you overclock the phone. BTW, make sure you have your air conditioner run at 60F and blowing directly on the phone, otherwise it will go up in smoke.

 

And I will condemn Apple if they did the same thing.

How could you possibly know what websites I look at? What exactly do you think my point is buddy? I don't think you are reading my posts.. I even said that by some great chance these tests are from complete unbiased phone saints. That they are totally accurate. And there is no agenda whatsoever at all not even to get more people to read their quaint site.. Unless they are the exact same benchmark tests using the same exact test that Sammy used on all the variations of the s4. And time after time they got different results. These prove nothing. Because I'll get 15 other tests saying no you used a different set of benchmarks.. You used these not those... The relative humidity wasn't set at exactly 12.25 percent and the temperature of the room exactly 88 degrees...

 

The only truth of the matter is that every company will give the best numbers for each of their devices... And there will always be ass holes out there saying no that ain't right. That benchmarks on the whole are not realistic measures of a phone. And each phone will be different for every person because of setting variations and apps used. Benchmarks are interesting bragging for companies only because the uneducated masses are impressed with raw numbers.. Oh it clocks at this speed my phone is 5 percent faster than yours it's my screen is bigger than yours.. I got more megapixels than you...

 

Trying to keep up with the Jones might as well measure your Johnson at least that's a tangible measure but like with your Johnson it's not the size.. But how you use it.. And you can have pissing contest till your standing in a gallon of piss. It all comes down to what works for you.. Not silly tests done in ideal conditions

Link to comment
Share on other sites

I think 2 different topics are getting confused here.

1. The validity of benchmarks.

2. Samsung intentionally skewing the results.

 

 

 

 

I agree im not going on the idea that anyone is skewing the result or more to the fact lying about any results. i believe that depending on the outcome you are looking for you will either get the results or post something that says that.  i could care less if someone claims samsung..apple...htc...motorola.. sony etc etc etc...  intentionally skewed the results.. every company puts their best foot forward... does that mean they intentionally put something out there that was absolutely false?  that they LIED..??. thats subjective.  i dont buy it.  did they get one really good result under a certain condition? and then post it maybe..  who knows. unless you do the exact benchmark test they did. under the exact same condition as they did. its impossible to say that anyone is lying. 

 

exaggerated sensational results now thats what you usually get in benchmarks either by the companies or those people that try to disprove it.. or those fanboys that pick through each looking for the measuring tape that says their phone is the biggest and baddest and the other guys phone is just to puny for them... 

Link to comment
Share on other sites

I think 2 different topics are getting confused here.

1. The validity of benchmarks.

2. Samsung intentionally skewing the results.

 

I am of the camp that benchmarks do very little to show how a phone will perform in real life conditions. The only way to know if a phone is "fast" enough for YOU is to use the phone under YOUR normal usage patterns. You have 14 days to test drive the phone. Use them.

This is exactly it. Benchmarks are synthetic, by nature. They're "raw performance". This isn't some kind of crazy optimization, it's just overclocking the device (sorta) under certain conditions. That's all. This isn't driver "optimizations" or "lying", it's literally the hardware performing at the maximum capable for the hardware. It would be invalid or lying if it was somehow reporting higher than the hardware could physically go. The title of this thread should frankly be changed.

 

But like I was trying to say before I sidetracked myself, benchmarks often have nothing to do with real-life performance. I can't think of any case I've been playing an Android game and thought "man, if only my GPU was faster, this wouldn't run so bad". I'm sure it happens, but 99% of what I do on my phone (and what I've seen most people with high end phones do) can't use anything close to "benchmark" power. Mostly it's Facebook and Candy Crush (ugh), if my customer are any indication.

Link to comment
Share on other sites

This is exactly it. Benchmarks are synthetic, by nature. They're "raw performance". This isn't some kind of crazy optimization, it's just overclocking the device (sorta) under certain conditions. That's all. This isn't driver "optimizations" or "lying", it's literally the hardware performing at the maximum capable for the hardware. It would be invalid or lying if it was somehow reporting higher than the hardware could physically go. The title of this thread should frankly be changed.

 

But like I was trying to say before I sidetracked myself, benchmarks often have nothing to do with real-life performance. I can't think of any case I've been playing an Android game and thought "man, if only my GPU was faster, this wouldn't run so bad". I'm sure it happens, but 99% of what I do on my phone (and what I've seen most people with high end phones do) can't use anything close to "benchmark" power. Mostly it's Facebook and Candy Crush (ugh), if my customer are any indication.

I guess where I disagree with you is that the benchmarks are giving out the impression of Software optimization that just isn't necessarily true.

My feeling of deception comes from Samsung playing on the tech ignorance of the masses that will see the benchmark comparisons on reviews and give Samsung credit for software and hardware optimization that is only an accurate portrayal of the phone while the benchmark is running.

Like I said earlier, I am not sure if this is Samsung being "bad", but is definitely a little sleight of hand. It creates a sense of what the phone can do out of the box, when in reality you would have to overclock the phone to actually get the performance that the benchmark made you think the phone was capable of.

All that said I completely see where you are coming from.

 

Sent from my SPH-L710 using Tapatalk 4 Beta

 

 

Link to comment
Share on other sites

I guess where I disagree with you is that the benchmarks are giving out the impression of Software optimization that just isn't necessarily true.

My feeling of deception comes from Samsung playing on the tech ignorance of the masses that will see the benchmark comparisons on reviews and give Samsung credit for software and hardware optimization that is only an accurate portrayal of the phone while the benchmark is running.

Like I said earlier, I am not sure if this is Samsung being "bad", but is definitely a little sleight of hand. It creates a sense of what the phone can do out of the box, when in reality you would have to overclock the phone to actually get the performance that the benchmark made you think the phone was capable of.

All that said I completely see where you are coming from.

 

Sent from my SPH-L710 using Tapatalk 4 Beta

 

 

Well, the article itself says that the performance boost happens during:

 

 

Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.

So, it's hardly just the benchmarking apps where it happens. The only issue here is that there's no user control, and it's not on all the time. It's really much ado about nothing. They chose to have higher performance when benchmarking and certain day-to-day tasks, but longer battery life for games, which is when it will drain the most. That's it, when you get down to it. The implications of malice are mostly only in the OP, nowhere else.

  • Like 1
Link to comment
Share on other sites

Well, the article itself says that the performance boost happens during:

 

 

So, it's hardly just the benchmarking apps where it happens. The only issue here is that there's no user control, and it's not on all the time. It's really much ado about nothing. They chose to have higher performance when benchmarking and certain day-to-day tasks, but longer battery life for games, which is when it will drain the most. That's it, when you get down to it. The implications of malice are mostly only in the OP, nowhere else.

I agree, the thread title is a bit sensationalist.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • large.unreadcontent.png.6ef00db54e758d06

  • gallery_1_23_9202.png

  • Posts

    • My s24 Ultras auto updated to April 1 security patch over night. Not on wi-fi.  Play system remains at March 1. No notice that update occurred on screen.
    • Got the latest update. On my S24 ultra the NR-ARFCN display remains constant (which is inaccurate according to diags) while n41 and 38 toggle back and forth.  I updated then rebooted before testing. Two sims active. My security patch level updated automatically to April 1 before this update. Diags sent while n38 was displaying.
    • Was able to install the March 1 Android security patch. Seems slightly more accurate with 5g ca band id, but can not swear by it. Updated google play system update through the software information screen to March 1. *#73# still works. Froze updates waiting on SCP update beta to fix n41 showing as n38.
    • Just installed it. Thanks for the info.  71 mb mar 1st date.
    • There's a permit for a new 47 story building at 205 Montague St in Downtown Brooklyn. The problem is that  T-Mobile eNB 48352 is on the building next door and this new building will block two out of 3 sectors of the site. For reference, the new building will be roughly as tall as 16 Court St which is right across the street. This site is the primary site covering Cadman Plaza so I wonder what the plan is. Will they just try to change sector placement, move to a different building, or will this just speed up the conversion of the Sprint site at 25 Monroe Place?
  • Recently Browsing

    • No registered users viewing this page.
×
×
  • Create New...