A single modern AI GPU consumes up to 3.7 MWh of power per year — GPUs sold last year alone consumed more power than 1.3 million homes

Nvidia data center GPUs
(Image credit: Nvidia)

Today's most powerful new data center GPUs for AI workloads can consume as much as 700 watts apiece. With a 61% annual utilization, that would account for about 3,740,520 Wh or 3.74 MWh per year per GPU, fueling concerns about the availability of power and environmental impacts, especially when we zoom out and look at the number of total GPUs sold last year alone.

Nvidia sold 3.76 million data center GPUs last year, accounting for 98% of the market. Adding the remaining 2% from Intel, AMD, and other players will account for over 3,836,000 GPUs delivered to data servers in 2023.

Multiply that number by the total data center GPU deliveries last year, and you get 14,348.63 GWh of electricity used in a year. To put this in context, the average American home uses 10,791 kWh per year, meaning data center GPUs sold last year consume the same amount of power that 1.3 million households use annually. If we look at this at the state level, the California Energy Commission reported that the state produced 203,257 GWh in 2022, meaning data center GPUs consume about 7% of the state’s annual production.

However, you should remember that these are just data center GPUs. The data here does not include the CPUs, cooling systems, and other equipment data centers need to run properly. For example, most RTX 4090s recommend a minimum 850-watt power supply, with some requiring 1,000 or even 1,200 watts. If we go by the minimum, servers and data centers built last year require over 25 GWh annually. These numbers do not even include data centers from 2022 and older and do not consider the many more coming online this year.

Industry analysts have estimated that the data center GPU market will grow by 34.6% year-on-year until 2028, meaning we will likely see more data center GPUs pumped out in the coming years. Furthermore, Nvidia’s next generation of AI GPUs are expected to draw more power than the current 700-watt H100. Even if data center computers retain their power consumption in the coming years (they won’t), power demand for data centers should increase proportionally with the market’s growth.

This unprecedented rise of data centers is raising concerns about our power infrastructure. In fact, the U.S. government is already in talks with tech companies about their AI electricity demands, especially as these new data centers could put undue pressure on the grid. Meta founder Mark Zuckerberg even says that limited power will constrain AI growth, especially as Enerdata noted that global power production only rose by 2.5% per year in the last decade.

Nevertheless, tech companies are not blind to this issue. Microsoft, traditionally a software company, is even looking to invest in small modular nuclear reactors for its data centers. This is especially important as it partnered with OpenAI to build a $100 billion AI supercomputer, which would definitely require a ton of power.

The rise of AI in our data-driven society means we need a lot of electricity to power our computing requirements. Furthermore, we mustn’t forget other upcoming technologies that also need a lot of juice, like EVs. Unless we find a way to develop chips (and motors) that deliver more power while simultaneously consuming less power, we’ll likely have to add more power production facilities and upgrade the supporting infrastructure to deliver them where needed.

Jowi Morales
Contributing Writer

Jowi Morales is a tech enthusiast with years of experience working in the industry. He’s been writing with several tech publications since 2021, where he’s been interested in tech hardware and consumer electronics.

  • bigdragon
    Hey, we don't need to be doing to my utility bills what has happened to my grocery bills. Stop!

    On a more serious note: we need a breakthrough in power generation soon. There has to be a more efficient alternative to the turbine that so many current high capacity power solutions rely upon. Modifying laws to encourage the installation of solar and wind solutions onto as many buildings as possible is only a start. We need something more.
    Reply
  • edzieba
    Whether AI or FEA simulation or big-ol-databases, datacentres have always been limited by grid power availability, then cooling capacity, (closely related to power availability but with some extra constraints), then by rack volume, in that order. If your individual component elements consume more power each, that typically means your rack utilisation ends up less dense rather than your datacentre consuming more power, because new grid connections and new water supplies are very hard to install, typically because you already built out the largest of each that was practical when you built the datacentre under exactly that knowledge.
    Reply
  • Eximo
    I imagine most datacenters use solar to offset cost of operation these days. But also consider that all those data centers have backup generators and battery backup. And I don't necessarily mean UPS in each rack, but whole building battery backups.

    The datacenter I used to work at had a battery room with a few tons of lead acid batteries that acted as a buffer while the generator kicked on. Though it was undersized and obsolete even when I was there.

    Based on some simple numbers. There is approximately 145 GW of available commercial rooftop space unused for solar panels in the US. That is roughly 10% of all power generated for the US. You could easily double that figure by considering similar sized commercial parking lots as well, just need to build out car shelters and put solar panels on those. (Plus side, weather covering for parking, and potential EV charging stations)

    But then you look at untapped land, and despite the energy transportation losses, it could be done. There is also agrisolar where you plant shade friendly crops between rows of solar panels, which means doing it with farmland that is more local to where people live. Maybe toss in some nuclear to cover any deficiencies.
    Reply
  • ThomasKinsley
    Move over EVs. We found a shinier powerhungry toy.
    Reply
  • Findecanor
    bigdragon said:
    On a more serious note: we need a breakthrough in power generation soon.
    Not only is that unrealistic.
    We would need a breakthroughs in power distribution as well.

    Just the legal and economic wrangling before actually starting to build a new stretch of a national power grid could take twenty years. (I once had a brief stint at a national power grid operator, that's how I know)
    Reply
  • hotaru251
    bigdragon said:
    we need a breakthrough in power generation soon
    we already have it.
    nuclear energy.

    its extremely reliable and only downside really is the rare event of type of accident Japan or Chernobyl. (these risks go up in war time as they become prime targets)

    nations could easily build more over next few decade. bonus side effect is it would be able to replace fossil fuels (the ppl in that business will fight to the death to make nuclear seem bad as once its adopted they lose all business) which in turn also help planet & would be a step towards combating global warming.

    theres around 440 currently running in just 30ish nations and they alone produce around 10% of total electricity produced by entire world. Now imagine if you scaled that we'd have more power than ever needed. (especially as w/ it you can also use solar, wind, & water along side em)

    On other subject thoguh I do think those companies who use over a certain amount of a nations total power should have to help fund increasing the overall system as their greed in it effects everyone else sharing it.
    Reply
  • Flayed
    bigdragon said:
    Modifying laws to encourage the installation of solar and wind solutions onto as many buildings as possible is only a start. We need something more.
    I'm not sure that the power companies really want this to happen as they want to sell you power not have you generate it yourself.
    Reply
  • USAFRet
    bigdragon said:
    Modifying laws to encourage the installation of solar and wind solutions onto as many buildings as possible is only a start.
    Solar, yes.
    Wind on buildings, maybe not so much.

    The big spinning whatevers can create a big vibration resonance in the building structure. Not fun to live in.
    This was seriously considered for apt buildings in NYC, until that vibration aspect came up.
    Reply
  • Notton
    The technology already exists
    Molten Salt Reactor
    Thorium Breeding Reactor
    Small Modular Reactor
    Ocean wave energy
    Seawater cooling
    etc.

    You also have plain old solar, wind, hydro, and nuclear fission that is good enough to keep up with demands.

    The "breakthrough" needs to happen at a different area. The will aspect of those in charge.
    And by that, I mean you will have to pitch the idea to the three corporations in a trenchcoat.
    Reply