Earlier this month Intel and Principled Technologies attracted controversy with their benchmarks comparing the new Core i9-9900K with AMD's Ryzen 7 2700X. (More on the new processor, which is the fastest gaming CPU we've ever tested, can be found in our review.) Now AMD has responded to the issue in new slides outlining its complaints with the first benchmark and the retest as well as its benchmarking best practices.
Make no mistake: there's effectively zero chance AMD released these slides the same day Intel lifted the embargo on Core i9-9900K reviews by accident. The company had ample opportunity to release these slides when the benchmarks first attracted controversy, or when the retest was published, instead of releasing them on the same day as Intel's new processor release.
But that doesn't mean AMD's complaints are invalid. The company has many of the same issues with Principled Technologies' initial benchmarks as most enthusiasts--the first test limited the 2700X's cores, had "questionable memory configurations," and was subject to conditions that favored Intel's processor over AMD's. That doesn't offer an unbiased comparison; it just gives Intel figures it can tout in press materials.
AMD's slides also take issue with the retest of these benchmarks. The company said it had the following concerns about those new benchmarks:
- Unclear Multicore Enhancement (or equivalent) setting on Z390 systems
- Suspect memory configurations (timings, transfer rates, capacity, derated OE DIMM specs)
- Unaddressed thermal environment disparities
- Unaddressed GPU assortment and performance deltas
- Unaddressed sample size, collection, and selection methodologies
- Unaddressed Z370 C-state configuration
The company raised these issues despite the retest showing double-digit performance improvements over the initial benchmarks in Far Cry 5, Forza Motorsport 7, Ashes of the Singularity, and Assassin's Creed Origins at 1080p. Addressing those outstanding concerns might allow the 2700X to compare even more favorably to the Core i9-9900K, but our own benchmarks show that Intel's latest still outperforms AMD's CPUs.
AMD also used the chance to share its "benchmarking best practices" for "consistent, accurate, repeatable" results. We suspect neither Intel nor Principled Technologies will heed the unsolicited advice.
Conversely, AMD has also contracted with Principled Technologies in the past to promote its EPYC data center chips.
Still, the entire back-and-forth shows the hazards of relying on vendor-supplied benchmarks. AMD was right to call out Intel and Principled Technologies for their setup and for touting the new Core i9-9900K's performance based on faulty comparisons. But--and, yes, we're clearly biased here--it's typically better to wait for impartial reviewers to get their hands on new hardware than to trust companies' performance claims.
