Demand for GPUs is unquestionably ramping up again as prices eclipse $70,000 per GPU in some China locales (that's for a data center H800 GPU), and leaders in the U.S. computing industry are taking to social media to complain that cloud-based GPU resources are fully booked and GPU hardware supplies are all reserved for the year ahead. Drastic times call for drastic measures, and we are beginning to see evidence that GPUs that should be heading to home desktop PC rigs are instead being snapped up by the AI industry players.
Naturally, gamers and enthusiasts will be worried about a repeat of the cryptomining craze, which decimated consumer GPU supplies: Are the crypto-bros of old destined to be replaced by the AI-bros — snapping up our precious gaming GPUs?
The first solid evidence of AI-focused businesses buying up consumer GPUs comes from a boast Tweeted by iconic IT hacker and entrepreneur George Hotz (AKA geohot).
AMD (unlike Qualcomm) is a joy to buy chips from. 7.38 PFLOPS of compute was delivered today, with exaflops more to come.Nice to see them more active on the ROCm GitHub, we are working on the layer above @__tinygrad__ pic.twitter.com/5yE11TVA5FAugust 1, 2023
Above, if you expand the tweet, you can see Hotz, who is currently leading an automated AI-driven driving assistance business called Comma AI, talk about buying up cases of AMD gaming GPUs.
The image shows a partly unpacked purported "7.38 PFLOPS of compute." We can clearly spy an XFX MERC 310 GPU in the photo, and the box reads "AMD Radeon RX 7900 XTX." That's an XFX design based around AMD's flagship consumer GPU and the second-placed Best Graphics Cards for Gaming in 2023 in our frequently updated roundup.
Our calculations from the Hotz statement and image are that there are 60 GPUs in this "7.38 PFLOPS" batch (based on FP16 performance). Each of these RX 7900 XTX cards costs $979.99 on Amazon, at the time of writing. Thus, Hotz apparently just splashed out around $60,000 on this modest stack of GPU power.
Obviously this is a mere drop in the ocean, as the Comma AI founder and president says that there are "exaFLOPS more to come." For reference, one exaFLOPS is 1,000 petaFLOPS of performance. If we assume that Hotz aims to buy up to 7.38 exaFLOPS of GPU compute (1,000 x the first batch), he may have budgeted ~$60 million for this consumer graphics card buying spree. Or perhaps only ~$20 million, if he's only after around 2 exaFLOPS of AI compute.
One interesting aside with the above purchase of AMD GPUs is that certain AI workloads can use a lot of memory. While paying nearly $1,000 per GPU for an AMD 7900 XTX that packs 'only' 122.88 teraFLOPS of FP16 number crunching prowess might seem odd, given the RTX 4060 costs $300 and can provide around 121 teraFLOPS of FP16 compute (with sparsity), the AMD GPU provides three times as much VRAM. So, if the workloads need memory, you'd need three RTX 4060 cards or two RTX 4070 cards to get the same 24GB — plus the servers to hold all those GPUs.
The Tweeted statement from George Hotz isn't the first warning sign we have seen of AI-industry demand for GPUs heating up. A few days ago, we reported on the rapid inflation of GPUs in China. Fears of further US sanctions on China are thought to have helped stoke up prices of accelerators like the A800 and H800 GPUs. A few weeks earlier, the price of an A800 jumped 20% almost overnight due to impending sanction fears.
Given other recent indicators, we can clearly see that the rise of AI is having a profound impact on the supply of compute-focused GPUs. Now the only question is how far that insatiable demand will bleed over to gaming GPUs, too.
Stay on the Cutting Edge
Join the experts who read Tom's Hardware for the inside track on enthusiast PC tech news — and have for over 25 years. We'll send breaking news and in-depth reviews of CPUs, GPUs, AI, maker hardware and more straight to your inbox.
Mark Tyson is a Freelance News Writer at Tom's Hardware US. He enjoys covering the full breadth of PC tech; from business and semiconductor design to products approaching the edge of reason.
Great, here we go again.Reply
@#%$#T#$ @#$%#@!%@!# GD!#$%!$#GRE#G!#$T$#@~!G$#!gReply
Excuse my language.
I guess NVIDIA, AMD and INTEL are going to revised their product segmentation for gaming and compute GPU at some point. Or AMD who provides more GDDR compared to NVidia is going to be happy to sell palettes of cards to AI people, no matter the effect on their more expensive compute cards sales. The 7900XTX 24 GB is indeed a good cheap deal for AI people. Anyway, it is not the first time nor the last that the pesky "consumers" use products (hardware of software) for performing operations the products were not designed for.Reply
even the companies dont want to touch the 4060's.Reply
Lol. I told everyone this was coming. Just wait until crypto explodes again along with AI. I just bought 4 4090's so I'm all ready to go. You all excited to pay $6000 for a 4090. I can't wait <3Reply
I’ve been buying gpus to use for my ai farm for quite some time, the boom is almost here. At least this time I’m ahead of the ball game.Reply
The promise of AI has been around for decades but has only been limited to the technology of the time. Today we have the technology between processing power and robotics engineering to make things even a decade ago seem impossible possible (especially around AI driven robotics). This has also increased demand and created new business need markets that didn't exist even five years ago.Reply
So far though, I don't see anything that points to AI demands even remotely coming close to the GPU gold-rush like demand insanity like we saw for crytpocurrency booms and crashes between 2010-2022. But time will tell. The only thing I'm worried about as I sit looking at my 2 year old EVGA RTX 3080 Ti is how much longer it will last for my needs (if it doesn't die on me), and if AMD is going to produce a high end Nvidia competitor to go up against their RTX 5080 next year.
Time to buy stocks for Nvidia and AMD!
and move the new GPU tree more years in the future. They will be more expensive then, but hopefully little bit faster also..
Damn! ai is the new mining 😭Reply
Gonna be stocking up on my popcorn for when the techie forums go wonkers if this comes into play... :cool:Reply