Sign in with
Sign up | Sign in

BAPCo Bites Back at AMD's Departure

By - Source: BAPCo Email | B 40 comments

BAPCo responded to AMD's allegations about SYSmark 2012 and the benchmark's alleged favoritism for another CPU manufacturer.

Wednesday morning we received a statement from Business Applications Performance Corporation (BAPCo) in regards to Tuesday's report about the departure of AMD. In a nutshell, AMD didn't agree with the direction SYSmark 2012 was taking, so the CPU manufacturer decided to leave BAPCo and retract its endorsement for the benchmark. On Wednesday BAPCo decided to respond.

"BAPCo is a non-profit consortium made up of many of the leaders in the high tech field, including Dell, Hewlett-Packard, Hitachi, Intel, Lenovo, Microsoft, Samsung, Seagate, Sony, Toshiba and ARCintuition," the company said in an emailed letter. "For nearly 20 years BAPCo has provided real world application based benchmarks which are used by organizations worldwide.  SYSmark 2012 is the latest release of the premiere application based performance benchmark.  Applications used in SYSmark 2012 were selected based on market research and include Microsoft Office, Adobe Creative Suite, Adobe Acrobat, WinZip, Autodesk AutoCAD and 3ds Max, and others."

"Advanced Micro Devices (AMD) was, until recently, a long standing member of BAPCo.," the consortium continued. "We welcomed AMD's full participation in the two year development cycle of SYSmark 2012, AMD's leadership role in creating the development process that BAPCo uses today and in providing expert resources for developing the workload contents.  Each member in BAPCo gets one vote on any proposals made by member companies.  AMD voted in support of over 80% of the SYSmark 2012 development milestones, and were supported by BAPCo in 100% of the SYSmark 2012 proposals they put forward to the consortium."

"BAPCo also notes for the record that, contrary to the false assertion by AMD, BAPCo never threatened AMD with expulsion from the consortium, despite previous violations of its obligations to BAPCo under the consortium member agreement.

"BAPCo is disappointed that a former member of the consortium has chosen once more to violate the confidentiality agreement they signed, in an attempt to dissuade customers from using SYSmark to assess the performance of their systems.  BAPCo believes the performance measured in each of the six scenarios in SYSmark 2012, which is based on the research of its membership, fairly reflects the performance that users will see when fully utilizing the included applications."

Tuesday AMD hinted that SYSmark 2012 doesn't provide "clear and reliable measurements" and is "misleading" thanks to the use of unrepresentative workloads that seemingly favors one particular non-AMD CPU manufacturer. AMD's Nigel Dessau even said that AMD tried to work within the BAPCo consortium in order to get the next-generation benchmark, SYSmark 2012, correct. He also stressed that it needed to be open, transparent and processor-neutral, but apparently the company went unheeded.

"Our good intentions were met with an outcome that we believe does a disservice to the industry and our customers," he said. "We weren’t able to effect positive change within BAPCo, and the resulting benchmark continues to distort workload performance and offers even less transparency to end users."

Discuss
Ask a Category Expert

Create a new thread in the News comments forum about this subject

Example: Notebook, Android, SSD hard drive

This thread is closed for comments
Top Comments
  • 19 Hide
    Flameout , June 22, 2011 11:35 PM
    blah blah blah it's funny how they just talk about other bs and not actually respond to whether their software is biased. with the muscle intel has, is it really any surprise that amd always gets the sh1t end of the stick
  • 18 Hide
    KT_WASP , June 22, 2011 11:46 PM
    This all rings hollow, especially since independent third parties have proven that the software is biased. What we have here, is, a poor attempt at damage control.

    Move along, nothing to see here.....Just like their benchmarking software.
  • 16 Hide
    sykozis , June 23, 2011 12:07 AM
    I got a good laugh at this...
    Quote:
    In the CRN Test Center, we agree; we gave up using SysMark years ago, and for some of the same reasons.

    Source: http://www.crn.com/news/applications-os/231000110/amd-to-bapco-we-quit.htm

    Seems BAPco is simply trying to "pull the wool over our eyes" so to speak.... It was also found by other sites, that by changing the CPUID, you could effectively alter the benchmark results. If that is actually true, it would seem that BAPco is trying to work as a marketing team for Intel.
Other Comments
    Display all 40 comments.
  • 5 Hide
    Ciuy , June 22, 2011 11:22 PM
    Didnt even know such a program (SYSmark 2012 oe 2011 or whatever) existed :o .

    Wont know either now, i dont have an Intel cpu .
  • 19 Hide
    Flameout , June 22, 2011 11:35 PM
    blah blah blah it's funny how they just talk about other bs and not actually respond to whether their software is biased. with the muscle intel has, is it really any surprise that amd always gets the sh1t end of the stick
  • 3 Hide
    Wisecracker , June 22, 2011 11:37 PM
    PassMark 'CPU Marks' sucks worse that BAPCo 'Sysmarks' It's a synthetic test race to the bottom between those two.

    And for those with a little spare cash laying around, you can buy your way into BAPCo for 20-large.
  • 18 Hide
    KT_WASP , June 22, 2011 11:46 PM
    This all rings hollow, especially since independent third parties have proven that the software is biased. What we have here, is, a poor attempt at damage control.

    Move along, nothing to see here.....Just like their benchmarking software.
  • -4 Hide
    dogman_1234 , June 22, 2011 11:59 PM
    Slap...from a slap.

    Seems like AMD has been getting crap we are not getting to understand. If so, i wouldn't doubt the Intel Cousin would leave because of its bigger cousin...Intel.
  • -5 Hide
    dalauder , June 23, 2011 12:03 AM
    Well now I'd like Tom's to run a test of BAPCo (stupid name, btw.) benchmarks. Tom's Hardware, like myself, is sympathetic to the underdog AMD, so it might not be best to risk hurting AMD if the benchmark makes them look bad while being accurate.

    The only way I really see Sysmark 2012 being biased is if the home user workloads use more than one or two threads since very few typical user applications are multi-threaded. Other than that, the benchmarks should be faster on whatever CPU is faster.

    I'm pretty sure 3dsmax and AutoCAD are faster on current non-AMD CPUs. We are talking about Intel, right?
  • 16 Hide
    sykozis , June 23, 2011 12:07 AM
    I got a good laugh at this...
    Quote:
    In the CRN Test Center, we agree; we gave up using SysMark years ago, and for some of the same reasons.

    Source: http://www.crn.com/news/applications-os/231000110/amd-to-bapco-we-quit.htm

    Seems BAPco is simply trying to "pull the wool over our eyes" so to speak.... It was also found by other sites, that by changing the CPUID, you could effectively alter the benchmark results. If that is actually true, it would seem that BAPco is trying to work as a marketing team for Intel.
  • -2 Hide
    rohitbaran , June 23, 2011 12:24 AM
    BAPCo should be renamed BarkCo!
  • 7 Hide
    eddieroolz , June 23, 2011 12:39 AM
    The howls of a lost dog. AMD was right in departing.
  • 8 Hide
    Anonymous , June 23, 2011 4:00 AM
    Hmmm - no comment by BAPCo that Nvidia & VIA also just quit BAPCo at the same time for the same reasons
  • 3 Hide
    DjEaZy , June 23, 2011 4:04 AM
    ... but many use irrelevant synthetic benchmarks or don't use benchmarks that show the power of GPGPU... sometimes i see tests for a system like mine and wonder, where come the low results from... my system behave much faster... in a real world applications... but that is, because i use a GPU too for some heavy operations... it's much faster...
  • -1 Hide
    masterofevil22 , June 23, 2011 4:24 AM
    Software in general is optimized for Intel cpu's bc THEY HAVE GREATER MARKET SHARE and quite simply a larger percentage of software developers' end users are going to see better performance by doing this. That's just the way it is.
  • 3 Hide
    beenthere , June 23, 2011 5:20 AM
    I fully support AMD's position on the bogus BAPCo bench and will not use this bench in consideration for any future purchases as it is unreliable and disengenuous.
  • -9 Hide
    Anonymous , June 23, 2011 6:03 AM
    I am surprised soo many of you are backing AMD. They are soooo far behind Intel that they are trying to discredit a benchmark that shows this. Seeing as Adobe Creative Suite is part of this benchmark, AMD probably fought extremely hard to get rid of it because they don't have the newer SSE extensions; thus, Intel walks all over them (in Premiere Pro CS5). Just search PPBM5 and see how badly AMD does in rendering and encoding tests.
  • 0 Hide
    tajisi , June 23, 2011 6:08 AM
    The problem, as I understand it, is that supposedly everything is compiled for Intel chips and gives Intel an artificial advantage. Thus, apparently, every benchmark site that gives an Intel chip a better review or shows higher performance is a part of a greater conspiracy. Also apparently the sky is blue due to NASA sending too many blueberries and Smurfs into space. Same difference -- next time use chipmunks.

    I'd be just as doubtful if Intel claimed the same about AMD.
  • 5 Hide
    Anonymous , June 23, 2011 7:08 AM
    http://www.osnews.com/story/22683/Intel_Forced_to_Remove_quot_Cripple_AMD_quot_Function_from_Compiler_
  • 3 Hide
    noob2222 , June 23, 2011 7:16 AM
    dalauderWell now I'd like Tom's to run a test of BAPCo (stupid name, btw.) benchmarks. Tom's Hardware, like myself, is sympathetic to the underdog AMD, so it might not be best to risk hurting AMD if the benchmark makes them look bad while being accurate.The only way I really see Sysmark 2012 being biased is if the home user workloads use more than one or two threads since very few typical user applications are multi-threaded.


    That is the inherent problem with Sysmark 2007, whether they continued the "dual thread only" for 2012 would explain why AMD would leave. you don't make a cpu only to have benchmark software test 1/4 of the power that is available.

    Seriously tho, how much faster can microsoft office load.

    Person A .. wow this doctument loaded in 1/2 a second.
    Person B ... haha mine only took 1/4 second.

    Here is another interesting fact on how disappointing Sysmark 2012 is.
    Quote:
    Without newer browsers, HTML5 is basically untested by SM12, and while we understand that SM12 has been in development for a while, for something calling itself 2012 to include mostly 2010 applications feels out of place.


    What this tells me is that Sysmark (updated every 5 or so years) will only be up to date on current software .... never.
  • 2 Hide
    silverblue , June 23, 2011 8:25 AM
    specialk9I am surprised soo many of you are backing AMD. They are soooo far behind Intel that they are trying to discredit a benchmark that shows this. Seeing as Adobe Creative Suite is part of this benchmark, AMD probably fought extremely hard to get rid of it because they don't have the newer SSE extensions; thus, Intel walks all over them (in Premiere Pro CS5). Just search PPBM5 and see how badly AMD does in rendering and encoding tests.

    The thing is, I'm curious about the timing. Bulldozer does indeed have the broadest support for extensions out of any AMD and Intel CPU and would indeed sufficiently outperform previous AMD offerings. Is this anything like the compiler furore we had over the past years where all CPUs are equal but some are more equal than others...?

    I'm afraid I know very little on this subject so I'd like to know which tests performed particularly badly on AMD hardware. I do believe, however, that - going back to the subject of compilers - Intel were ordered to remove the code that purposefully hijacks performance on non-Intel CPUs.
  • 3 Hide
    aznshinobi , June 23, 2011 8:48 AM
    So Tom's, when you guys going to release an article about whether or not Sysmark is bias?
  • -8 Hide
    nebun , June 23, 2011 12:16 PM
    AMD is leaving because it can't keep up with intel....they should really just start making video cards
Display more comments