Elon Musk Buys Thousands of GPUs for Twitter's Generative AI Project

News
By Anton Shilov
published

Twitter is reportedly building a ChatGPT rival.

Nvidia
(Image credit: Nvidia)

Despite advocating for an industry-wide halt to AI training, Elon Musk has reportedly kicked off a major artificial intelligence project within Twitter. The company has already purchased approximately 10,000 GPUs and recruited AI talent from DeepMind for the project that involves a large language model (LLM), reports Business Insider.

One source familiar with the matter stated that Musk's AI project is still in its initial phase. However, acquiring a significant amount of additional computational power suggests his dedication towards advancing the project, as per another individual. Meanwhile, the exact purpose of the generative AI is unclear, but potential applications include improving search functionality or generating targeted advertising content. 

At this point, it is unknown what exact hardware was procured by Twitter. However, Twitter has reportedly spent tens of millions of dollars on these compute GPUs despite Twitter's ongoing financial problems, which Musk describes as an 'unstable financial situation.' These GPUs are expected to be deployed in one of Twitter's two remaining data centers, with Atlanta being the most likely destination. Interestingly, Musk closed Twitter's primary datacenter in Sacramento in late December, which obviously lowered the company's compute capabilities. 

In addition to buying GPU hardware for its generative AI project, Twitter is hiring additional engineers. Earlier this year, the company recruited Igor Babuschkin and Manuel Kroiss, engineers from AI research DeepMind, a subsidiary of Alphabet. Musk has been actively seeking talent in the AI industry to compete with OpenAI's ChatGPT since at least February. 

OpenAI used Nvidia's A100 GPUs to train its ChatGPT bot and continues to use these machines to run it. By now, Nvidia has launched the successor to the A100, its H100 compute GPUs that are several times faster at around the same power. Twitter will likely use Nvidia's Hopper H100 or similar hardware for its AI project, though we are speculating here. Considering that the company has yet to determine what its AI project will be used for, it is hard to estimate how many Hopper GPUs it may need. 

When big companies like Twitter buy hardware, they buy at special rates as they procure thousands of units. Meanwhile, when purchased separately from retailers like CDW, Nvidia's H100 boards can cost north of $10,000 per unit, which gives an idea of how much the company might have spent on hardware for its AI initiative.

Anton Shilov
Anton Shilov
Freelance News Writer

Anton Shilov is a Freelance News Writer at Tom’s Hardware US. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.

See more GPUs News
29 Comments Comment from the forums
  • PlaneInTheSky
    AI is the next scam.

    After decades of Tesla using "AI" and "machine learning", the cars are still horrendous at self-parking. Companies not relying on "AI" for parking have been able to properly park for years.

    nsb2XBAIWyA
    Reply
  • bniknafs9
    am i dumb or is AI the next Gen business , after Mining.
    Reply
  • JamesJones44
    bniknafs9 said:
    am i dumb or is AI the next Gen business , after Mining.

    Yes and no. The word AI has definitely become the hot keyword, just simply put AI in your company name, throw it out in a conference call or say your doing something (anything) with it and you will get a stock pop. Most companies that claim they are doing AI work aren't or are just simply leveraging existing ML tools to extend some existing functionality than doing anything that will move the needle sales wise.

    However, AI in computer vision has been a big move forward and AI in the form of LLM has also been able to do things that were difficult to do with lines of code (not impossible, but difficult and expensive). In this way AI is a next gen business, but with some caveats. However, companies jamming LLM into every product and calling it a revolution, well, that is largely just cashing in on the hype.

    In my personal opinion the companies that best leverage AI will be the winners, but I don't think their will be a single company that will get associated with AI the way things like search has with Google or Netflix with streaming. People are trying to do it with ChatGPT, but ChatGPT is just one type of AI, there are many many forms of AI/ML that are good at specific tasks and I think the idea of one model to rule them all is unlikely at this point in the cycle/evolution.
    Reply
  • hotaru251
    PlaneInTheSky said:
    AI is the next scam.
    its not.

    ai has uses just needs to mature properly.


    especially in fields like medical where theres a ton of info and having an ai use your given input to offer outputs can save you time.
    same for coding as bug testing your code is timely and ai doing it for you quickly is helpful.


    whatever reason Musk wants it for thoguh is for sure a waste of money, effort, & power.
    Reply
  • jkflipflop98
    AI properly applied is most certainly no scam. We've used it at work for years for all kinds of tasks.

    It's one of the next big steps in computing. We have insane amounts of compute power and access to almost all knowledge of the combined human race. It makes logical sense to use the former to exploit the latter and come up with trends and points and combinations that a human would have never figured out.
    Reply
  • InvalidError
    PlaneInTheSky said:
    AI is the next scam.
    I wouldn't say it is a scam but the AI-ification of some relatively simple things is just plain stupid. Parking is basic geometry. Using AI to find the spot, make sure it is safe to get in and monitor progress makes sense. Once all the necessary distances have been measured though, parking itself should be a pre-programmed sequence as long as tires don't slip and surrounding parked cars stay put.
    Reply
  • evdjj3j
    hotaru251 said:
    its not.

    ai has uses just needs to mature properly.


    especially in fields like medical where theres a ton of info and having an ai use your given input to offer outputs can save you time.
    same for coding as bug testing your code is timely and ai doing it for you quickly is helpful.


    whatever reason Musk wants it for thoguh is for sure a waste of money, effort, & power.

    I would love a service that I could upload an MRI or some other diagnostic imaging to and have an AI double check the human radiologist.
    Reply
  • domih
    evdjj3j said:
    I would love a service that I could upload an MRI or some other diagnostic imaging to and have an AI double check the human radiologist.

    It is already an industry which has been active for several years and is in full expansion. See for instance, among others, https://deepbio.co.kr/ or https://www.gleamer.ai/. There are 100+ hardware and software companies, using AI, working in radiology or pathology, etc.

    There are already quite a lot of hospitals or other health organizations using these solutions on site.

    The practitioners (e.g. radiologists) are not replaced by AI. They still are the ones interpreting the X-rays, CAT-scans, etc and writing the reports. The AI solutions are only helping by detecting issues that a human might miss. This is quite important in the case of cancers where the earlier the detection is the better it is to fight it.

    The NY Times and many other mainstream press have many articles about this industry. It does not make noise like ChatPGT but it is a serious business with massive financial weight.

    You just have to look for it to find it.

    You can also look for DICOM, a file format and a protocol used in radiology, dentistry, etc. If you see an X-Ray on a doctor or dentist monitor, it is probably a DICOM file.

    HTH
    Reply
  • russell_john
    We need to stop training AI (So I can catch up)

    - Elon Musk
    Reply
  • bit_user
    Hmm... Musk has business relationships that pose some interesting questions.
    As one of the OpenAI founders, can he no longer gain access to their technology, or does Microsoft now control too much of the company for him to have such pull?
    Considering Tesla's AI hardware sounds pretty impressive, why not arrange to buy some of theirs? Are there technological differences that significantly disadvantage it on running transformer networks?
    When big companies like Twitter buy hardware, they buy at special rates as they procure thousands of units.
    Yes... but, Nvidia is basically the single supplier of the hardware everyone wants to use. That gives them quite a bit of leverage, in any price negotiations.

    10k GPUs is a lot of money for a company that (I think) is still making losses. If we assume about $20k each (including the servers to host them), that's a cool $200M. Only about 1% of what he paid for Twitter, but probably a multiple of Twitter's annual hardware spend.

    Nvidia's H100 boards can cost north of $10,000 per unit,
    Somehow, I had a figure of $18k in mind. Not sure if I'm misremembering that or maybe the street price has shot way up since then. shopping.google.com shows prices anywhere from $28.5k to ebay prices of $43k or more.

    Then, I thought I'd see what Dell's list price is, so I popped over to dell.com and looked at the price of adding one to a PowerEdge R750xa. They want an absolutely astounding $86,250 per H100 PCIe card, and they make you add a minimum of 2 GPUs to the chassis!!! Having a decent amount of experience with Dell servers at my job, I know they like big markups for add-ons, but I'm still pretty stunned by that one.

    If you know anything about these, you're probably aware that the PCIe cards aren't even the best type of H100. What you really want are the SXM version. And a further irony is that a pair of the current H100's cannot even run GPT-3, which is why Nvidia recently announced a refresh of its H100 with more memory, due out in Q3.
    Reply