We all love a good story, and AMD's phoenix-like rise from the ashes of the CPU market is a great one. AMD is truly making a wonderful comeback, earning our praise for many of its recent chips, but we have to temper the widespread recent reports of AMD's fantastical CPU gains in the Steam Hardware Survey.
No, AMD hasn't made a 2.5X improvement in two years in the Steam Hardware Survey. You shouldn't pay attention to those reports, and we'll explain why. We'll also touch a bit on recent reporting surrounding the bogus Passmark "Market Share Report."
Steam's False Hardware Survey Results
The Steam Hardware Survey tracks a portion of Steam's 125 million active users, so the media often uses it to track the hardware and software that gamers use most frequently.
Simply put, by its own admission, Valve's Steam Hardware Survey was broken from September 2017 to May 2018, and the recent reporting outlines a 2.5X gain in AMD CPU usage on systems with Steam. Unfortunately, that reporting comes to that conclusion based on an increase from January 2018, which falls squarely in the middle of Steam's false survey results (as seen in the chart below). That means the recent reporting is wrong due to incorrect survey data.
During this time, the survey reported data from Chinese internet cafes that submitted multiple submissions per computer frequently, instead of yearly as the survey is designed. This happened so frequently on these Intel-powered systems that it skewed all Steam Survey results, including resolutions, operating systems, GPUs used, and so on. The results were so badly skewed that Valve announced it to the public.
However, even when the survey is working correctly, it isn't really indicative of the desktop PC market, which is where AMD is most competitive. Steam doesn't share much public information about its survey methodologies, such as how many systems it queries per year, or the percentage of laptops and desktop PCs in the surveyed data, so we can’t use these numbers as a direct comparison of AMD's penetration into desktop gaming rigs. Interesting data, sure, but it is irrevocably skewed due to laptops (which are ~60% of the overall CPU market) submitted into the same survey pool.
According to the survey, based on the correct reporting period beginning in April 2018, AMD's CPU use on Steam has gained roughly 3.48%. Again, this isn't particularly helpful data, but it highlights the massive divide between the reports and reality. Also, remember that these numbers quantify all of the processors in use worldwide, and considering the typical five- to seven-year refresh rate of desktop PCs, it includes an incredible amount of pre-Ryzen-era systems. That makes it hard to gauge current adoption rates accurately.
We turn to Mercury Research, the same agency that AMD and Intel (among many other semiconductor vendors) use for their competitive analysis reports, for real measurements of unit share. You can see those here. These numbers aren't nearly as exciting, but remember that each percentage point of gain in these numbers potentially equates to 100's of millions of dollars.
Passmark....Again.
Finally, a quick note on Passmark. We frequently see analysis of these results crop up in articles, with fantastic claims like AMD achieving 30% of the market. Unfortunately, the Passmark 'report' is even more misleading.
Passmark posts quarterly updates that outline the number of benchmark submissions the company has received, but due to a horribly-named and incredibly misleading chart and article title, many mistake the results as an indicator of AMD's market share. The results do not represent actual sales figures, and certainly do not represent market share. Instead, it quantifies the number of times that users run the Passmark benchmark. If you're overclocking your system and running the test repeatedly to gauge performance improvements, each test counts on this chart.
The chart is also updated daily, so you have to eliminate the final data point from consideration, as it is never complete.
We threw this graph together with Mercury Research's desktop PC numbers compared to Passmark. There is some loose correlation to increasing adoption of AMD CPUs over time, but the problem lies in the deltas. Passmark is off by 8% on average, but sees spikes as high as 13.2% that don't correlate to actual share increases. In fact, they've even occurred during declines in AMD's CPU unit share.
Bear in mind, we're comparing strictly desktop processor unit share data from Mercury Research to the Passmark results, while the latter can include test submissions from laptops and servers. As we see in 3Q17 and 3Q19, there are large unexplained deltas, though its rational to assume the 3Q19 Passmark data point is the result of new Ryzen users measuring their boost clock speeds during the recent controversy surrounding the issue. Notably, this errant data point is the one used for recent reports.
To be thorough, although we don't have as much historical data, we plotted the overall unit share numbers from Mercury to compare to the Passmark numbers. That doesn't help much.
In sum, neither Passmark or the Steam Hardware Survey are accurate indicators of CPU market share, especially if they are used to measure a single point in time. Broad trends, maybe, but not quarterly share figures.
