AMD Responds to Intel's 9th Gen Benchmarks

Earlier this month Intel and Principled Technologies attracted controversy with their benchmarks comparing the new Core i9-9900K with AMD's Ryzen 7 2700X. (More on the new processor, which is the fastest gaming CPU we've ever tested, can be found in our review.) Now AMD has responded to the issue in new slides outlining its complaints with the first benchmark and the retest as well as its benchmarking best practices.

Make no mistake: there's effectively zero chance AMD released these slides the same day Intel lifted the embargo on Core i9-9900K reviews by accident. The company had ample opportunity to release these slides when the benchmarks first attracted controversy, or when the retest was published, instead of releasing them on the same day as Intel's new processor release.

But that doesn't mean AMD's complaints are invalid. The company has many of the same issues with Principled Technologies' initial benchmarks as most enthusiasts--the first test limited the 2700X's cores, had "questionable memory configurations," and was subject to conditions that favored Intel's processor over AMD's. That doesn't offer an unbiased comparison; it just gives Intel figures it can tout in press materials.

AMD's slides also take issue with the retest of these benchmarks. The company said it had the following concerns about those new benchmarks:

  • Unclear Multicore Enhancement (or equivalent) setting on Z390 systems
  • Suspect memory configurations (timings, transfer rates, capacity, derated OE DIMM specs)
  • Unaddressed thermal environment disparities
  • Unaddressed GPU assortment and performance deltas
  • Unaddressed sample size, collection, and selection methodologies
  • Unaddressed Z370 C-state configuration

The company raised these issues despite the retest showing double-digit performance improvements over the initial benchmarks in Far Cry 5, Forza Motorsport 7, Ashes of the Singularity, and Assassin's Creed Origins at 1080p. Addressing those outstanding concerns might allow the 2700X to compare even more favorably to the Core i9-9900K, but our own benchmarks show that Intel's latest still outperforms AMD's CPUs.

AMD also used the chance to share its "benchmarking best practices" for "consistent, accurate, repeatable" results. We suspect neither Intel nor Principled Technologies will heed the unsolicited advice.

Conversely, AMD has also contracted with Principled Technologies in the past to promote its EPYC data center chips.

Still, the entire back-and-forth shows the hazards of relying on vendor-supplied benchmarks. AMD was right to call out Intel and Principled Technologies for their setup and for touting the new Core i9-9900K's performance based on faulty comparisons. But--and, yes, we're clearly biased here--it's typically better to wait for impartial reviewers to get their hands on new hardware than to trust companies' performance claims.

Create a new thread in the News comments forum about this subject
36 comments
Comment from the forums
    Your comment
  • dudmont
    AMD should focus it's energies on improving the IPC of it's chips. I want them to improve, not complain. If they were just a few percentage points behind in IPC, then the performance gap would become a non-issue and things like features and price would be paramount. As it is, they're still lagging by enough that one has to think about what they value more, price vs performance.
  • redgarl
    "But--and, yes, we're clearly biased here--it's typically better to wait for impartial reviewers to get their hands on new hardware than to trust companies' performance claims."

    WHAT?!!! Just buy it! Just buy the 9900k! Toms review of the 9900k is just saying so even if the thermal, power consumption and price isn't even took into consideration with a chokable 4.5/5.
  • John Pombrio
    AMD should have released their results before the reviews came out. Now it is immaterial what the preliminary results were as the reviewers results show that the results are still in Intel's favor, just not as much as the initial report made it out to be.

    As for AMD's bitter reply here, I remember when AMD published their own "benchmarks" for the Fury X two weeks before the independent reviews came out. The difference in their benchmarks and the independent benches were in about the same ballpark, 10% less than the actual results due to cherrypicking the settings for the Fury and using other settings on the GTX 980 Ti (or whatever else they were comparing it at). I learned that lesson well not to trust a company's benchmarks before the release.