Meta was one of the first companies to build its RISC-V-based chips for AI inference several years ago to cut costs and reduce reliance on Nvidia. Reuters reports that the company went one step further and designed (presumably with Broadcom's assistance) its in-house accelerator for AI training. If the chip meets Meta's goals, it may reduce its reliance on high-end Nvidia AI GPUs —such as H100/H200 and B100/B200—for training advanced large-language models.
Meta and Broadcom have taped out Meta's first AI training accelerator with TSMC; the latter produced the first working samples of these chips, and the partners have successfully brought up the unit, according to the report. By now, Meta has started with a limited deployment of the accelerator, assessing its performance before scaling up production and deployment. It is unclear whether Meta's engineers are running benchmarks on the new chip; it has already been deployed to make some useful work.
The chip's specifications are unknown, though typically, AI training chips use a design known as a systolic array. This architecture consists of a structured network of identical processing elements (PEs) arranged in rows and columns. Each unit handles computations involving matrices or vectors, and data flows sequentially through the network.
Latest Videos FromTom's Hardware
Custom RISC-V Accelerator For AI
Since the processor is designed for AI training — which means processing vast amounts of data — expect the processor to feature HBM3 or HBM3E memory. Considering that we are dealing with a bespoke processor, Meta defined its supported data formats and instructions to optimize die size, power consumption, and performance. As for performance, the accelerator has to offer competitive performance-per-watt characteristics with Nvidia's up-to-date AI GPUs, such as H200, B200, and possibly next-generation B300.
The chip is the latest addition to Meta's Meta Training and Inference Accelerator (MTIA) program. The program has faced various setbacks, including when development was halted at similar stages.
For example, discontinued its internal inference processor after it failed to meet its performance and power targets during limited deployment tests. This failure led Meta to shift its strategy in 2022, placing large orders for Nvidia GPUs to meet its immediate AI processing requirements.
Since then, Meta has become one of Nvidia's largest customers, acquiring tens of thousands of GPUs. These units have been critical in training AI models for recommendations, advertisements, and the Llama Foundation model series. Also, the green company's GPUs have been employed for inference processes, supporting interactions for over three billion daily users across Meta's platforms, according to Reuters.
Despite these challenges, Meta has continued advancing its custom silicon program. Last year, Meta began using an MTIA chip for inference tasks, and looking ahead, Meta's leadership has outlined plans to start using its custom chips for AI training by 2026. The plan is to gradually increase usage if the chip meets performance and power targets, which is a critical component of Meta's long-term goal to design more customized hardware solutions for its data center operations.
One interesting thing to note is that MTIA's accelerators for inference use open-source RISC-V cores. This enables Meta to customize instruction set architecture as it wishes to meet its requirements at its cadence, but on the other hand, it does not need to pay royalties to any third party. It is unclear whether MTIA's training accelerator is also based on the RISC-V ISA, but this is possible. If this is true, Meta might have developed one of the industry's highest-performing RISC-V-based chips.
