AMD Responds to Intel's 9th Gen Benchmarks

Earlier this month Intel and Principled Technologies attracted controversy with their benchmarks comparing the new Core i9-9900K with AMD's Ryzen 7 2700X. (More on the new processor, which is the fastest gaming CPU we've ever tested, can be found in our review.) Now AMD has responded to the issue in new slides outlining its complaints with the first benchmark and the retest as well as its benchmarking best practices.

Make no mistake: there's effectively zero chance AMD released these slides the same day Intel lifted the embargo on Core i9-9900K reviews by accident. The company had ample opportunity to release these slides when the benchmarks first attracted controversy, or when the retest was published, instead of releasing them on the same day as Intel's new processor release.

But that doesn't mean AMD's complaints are invalid. The company has many of the same issues with Principled Technologies' initial benchmarks as most enthusiasts--the first test limited the 2700X's cores, had "questionable memory configurations," and was subject to conditions that favored Intel's processor over AMD's. That doesn't offer an unbiased comparison; it just gives Intel figures it can tout in press materials.

AMD's slides also take issue with the retest of these benchmarks. The company said it had the following concerns about those new benchmarks:

  • Unclear Multicore Enhancement (or equivalent) setting on Z390 systems
  • Suspect memory configurations (timings, transfer rates, capacity, derated OE DIMM specs)
  • Unaddressed thermal environment disparities
  • Unaddressed GPU assortment and performance deltas
  • Unaddressed sample size, collection, and selection methodologies
  • Unaddressed Z370 C-state configuration

The company raised these issues despite the retest showing double-digit performance improvements over the initial benchmarks in Far Cry 5, Forza Motorsport 7, Ashes of the Singularity, and Assassin's Creed Origins at 1080p. Addressing those outstanding concerns might allow the 2700X to compare even more favorably to the Core i9-9900K, but our own benchmarks show that Intel's latest still outperforms AMD's CPUs.

AMD also used the chance to share its "benchmarking best practices" for "consistent, accurate, repeatable" results. We suspect neither Intel nor Principled Technologies will heed the unsolicited advice.

Conversely, AMD has also contracted with Principled Technologies in the past to promote its EPYC data center chips.

Still, the entire back-and-forth shows the hazards of relying on vendor-supplied benchmarks. AMD was right to call out Intel and Principled Technologies for their setup and for touting the new Core i9-9900K's performance based on faulty comparisons. But--and, yes, we're clearly biased here--it's typically better to wait for impartial reviewers to get their hands on new hardware than to trust companies' performance claims.

Nathaniel Mott
Freelance News & Features Writer

Nathaniel Mott is a freelance news and features writer for Tom's Hardware US, covering breaking news, security, and the silliest aspects of the tech industry.

  • dudmont
    AMD should focus it's energies on improving the IPC of it's chips. I want them to improve, not complain. If they were just a few percentage points behind in IPC, then the performance gap would become a non-issue and things like features and price would be paramount. As it is, they're still lagging by enough that one has to think about what they value more, price vs performance.
    Reply
  • redgarl
    "But--and, yes, we're clearly biased here--it's typically better to wait for impartial reviewers to get their hands on new hardware than to trust companies' performance claims."

    WHAT?!!! Just buy it! Just buy the 9900k! Toms review of the 9900k is just saying so even if the thermal, power consumption and price isn't even took into consideration with a chokable 4.5/5.
    Reply
  • John Pombrio
    AMD should have released their results before the reviews came out. Now it is immaterial what the preliminary results were as the reviewers results show that the results are still in Intel's favor, just not as much as the initial report made it out to be.

    As for AMD's bitter reply here, I remember when AMD published their own "benchmarks" for the Fury X two weeks before the independent reviews came out. The difference in their benchmarks and the independent benches were in about the same ballpark, 10% less than the actual results due to cherrypicking the settings for the Fury and using other settings on the GTX 980 Ti (or whatever else they were comparing it at). I learned that lesson well not to trust a company's benchmarks before the release.
    Reply
  • fiber4now
    It is the same with AMD it is always someone else's fault. I remember the launch of the 1800x when it performed on par with a Intel I3 of the time,

    it could not be AMD's fault most said but in truth it was lower IPC performance and interconnect latency. This forced the same individual who take exception to PT testing to insist upon Higher frequency system memory to be used in order to improve the 1800X performance.

    It is hard to imagine after being out of the CPU performance race for over a decade how being close without a single gaming victory would cause such ire. Unless most complaints come from AMD stock holders pretending to be gamers.
    Reply
  • elbert
    Given the price of the 9900K I bet AMD comes with a 2800x and tests of the 9900k vs the 12 core 2920X. Possible even a price change for the 2920X to compete directly with the 9900K.
    Reply
  • artk2246
    21415565 said:
    It is the same with AMD it is always someone else's fault. I remember the launch of the 1800x when it performed on par with a Intel I3 of the time,

    it could not be AMD's fault most said but in truth it was lower IPC performance and interconnect latency. This forced the same individual who take exception to PT testing to insist upon Higher frequency system memory to be used in order to improve the 1800X performance.

    It is hard to imagine after being out of the CPU performance race for over a decade how being close without a single gaming victory would cause such ire. Unless most complaints come from AMD stock holders pretending to be gamers.

    Do you have a link for that article? Even when it originally launched with its memory issues, I dont remember it ever performing near an I3, unless you were talking about purely single threaded workloads.
    Reply
  • anghellic
    Amd has no reason to complain everyone saw through the BS Intel pulled and principled technologies even with the 2nd test so no real reason for them to complain and the fact that there is a 20% price jump for Intel new CPU for a 10% performance gain
    Reply
  • anghellic
    I cannot wait for new amd 7nm cpu if they can accomplish it they will pull ahead of intel with no problem on market share
    Reply
  • kinggremlin
    Pot calling the kettle black.
    Reply
  • jimmysmitty
    21415748 said:
    Given the price of the 9900K I bet AMD comes with a 2800x and tests of the 9900k vs the 12 core 2920X. Possible even a price change for the 2920X to compete directly with the 9900K.

    I don't see them pushing a 12 core to the mainstream. Besides it drawing more power it would have to clock loser and AMD already has a clock deficit vs Intel. The normal power draw makes sense as does the torture test, most CPUs clocked to that high of a level will pull more power.

    And threadripper is meant to compete in the HEDT market. It should not be compared to LGA1151.

    And this response is, in my opinion, worthless. Mainly because AMD has done their own "touting". Every company does. They are never going to show them losing to the competition. When the FX 8150 launched they picked only tests that the CPU performed better than certain Intel CPUs. Its common practice.

    Intel does need to get pricing under control though.
    Reply