The TinyBox AI accelerator is now available on sale on the tinygrad website starting at $15k. This small 16.25-inch deep, 12U freestanding unit sports six AMD Radeon RX 7900XTX or Nvidia GeForce RTX 4090 GPUs designed for deep learning and AI acceleration applications. While the computer is relatively more expensive versus consumer gaming desktop PCs, it’s much more affordable than a single Nvidia H100 or AMD MI300X accelerator, which are priced between $10 to $40k apiece.
18 months in to the company, tinyboxes finally have a buy it now button! We have 13 in stock today, go to our website (link on @__tinygrad__) to buy one.The $15k tinybox red is the best perf/$ ML box in the world. It's fully networkable, so that's the metric that matters. pic.twitter.com/gFxc873Q1yAugust 26, 2024
tinygrad founder George Hotz shared the news on X about the availability of TinyBox, saying that they have 13 units in stock as of August 27. He also claims that it offers “the best perf/$ ML box in the world” and that it’s fully networkable. Although the company faced some challenges with the AMD GPUs it used for its systems, it was eventually able to find a solution to this problem. At the same time, it also added the option to use Nvidia GPUs instead to avoid AMD’s driver instability, although this comes at a 67% premium.
| Header Cell - Column 0 | red | green |
|---|---|---|
| TFLOPS | 738 FP16 TFLOPS | 991 FP16 TFLOPS |
| GPU Model | 6x RX 7900XTX | 6x RTX 4090 |
| GPU RAM | 144GB | 144GB |
| GPU RAM bandwidth | 5,760 GB/s | 6,050 GB/s |
| GPU link bandwidth | 6x PCIe 4.0 x16 (64 GB/s) | 6x PCIe 4.0 x16 (64 GB/s) |
| CPU | 32-core AMD EPYC | 32-core AMD EPYC |
| System RAM | 128 GB | 128 GB |
| System RAM bandwidth | 204.8 GB/s | 204.8 GB/s |
| Disk size | 4TB raid array + 1TB boot | 4TB raid array + 1TB boot |
| Disk read bandwidth | 28.7 GB/s | 28.7 GB/s |
| Networking | 2x 1 Gbe + open OCP3.0 slot (up to 200 Gbe) | 2x 1 Gbe + open OCP3.0 slot (up to 200 Gbe) |
| Noise | <50 dB, 31 low speed fans | <50 dB, 31 low speed fans |
| Power supply | 2x 1600W, 100~240V | 2x 1600W, 100~240V |
| BMC | AST2500 | AST2500 |
| Operating system | Ubuntu 22.04 | Ubuntu 22.04 |
| Dimensions | 12U, 16.25-inch deep, 90 lbs. | 12U, 16.25-inch deep, 90 lbs. |
| Rack mount | Freestanding or rack mount | Freestanding or rack mount |
| Driver quality | Acceptable | Great |
| Price | $15,000 | $25,000 |
Despite these seemingly high prices, the company reportedly already has 583 pre-orders for the system. And, with the drama with AMD’s GPUs behind it, it looks like that tinygrad is ready to go full steam ahead with the sale of its TinyBox AI accelerators. Companies have a choice between the more affordable AMD GPU or the Nvidia system. However, as the tinygrad said in a Tweet before, “If you like to tinker and feel pain, buy red. The driver still crashes the GPU and hangs sometimes, but we can work together to improve it.”
The factory is at full power! If you have a tinybox preorder and somehow haven't been contacted, reach out to support@tinygrad.orgSales will open to the public shortly. pic.twitter.com/xljpYU1HjvAugust 26, 2024
We'd like to believe that it has finally solved the issues with the AMD GPUs. Hence, it’s making it available to the market. But with the popularity of Nvidia in the AI accelerator space, it does make sense for the company to give this option, too. It’s just unfortunate, though, that the Intel Arc-powered TinyBox is still in the prototype stage and that it has no plans to ship it. It would’ve been nice if we had the option for that as well, so we hope that the company changes its mind in the future and starts shipping ‘blue’ models of the TinyBox.
