The tinybox pro is now up for preorder on the tinygrad website, offering two more GPUs, an extra AMD Genoa EPYC processor, and three times the system RAM as the original $15,000 TinyBox AI accelerator.
Although its $40k asking price is more than double that of the tinybox red, the pro version could hit 1.36 PetaFLOPS of FP16 computing with its eight RTX 4090s and 192GB of GPU RAM. While the tinybox pro is closer to the price of a single Nvidia H100 GPU, it’s still a more affordable alternative considering you’re getting a complete system versus just one graphics card.
tinybox prooooooooooooooooooo (preorder today) pic.twitter.com/tGBznXdrHyNovember 13, 2024
The company shared an image of the tinybox pro on X on November 13, saying it’s available for preorder today. The pro model seems only available in a rack-mounted case, with its specifications saying that it’s compatible with Supermicro rails. The spec table also doesn’t say how much noise its fans generate, only saying that it’s ‘Loud,’ which shows that it prioritizes performance over the aural comfort of its users.
| tinybox | red | green | pro |
|---|---|---|---|
| FLOPS | 738 FP16 TFLOPS | 991 FP16 TFLOP | 1.36 FP16 PFLOPS |
| GPU Model | 6x 7900 XTX | 6x RTX 4090 | 8x RTX 4090 |
| GPU RAM | 144GB | 144GB | 192GB |
| GPU RAM Bandwidth | 5,760 GB/s | 6,050 GB/s | 8,064 GB/s |
| GPU Link Bandwidth | Full PCIe 4.0 x16 (64 GB/s) | Full PCIe 4.0 x16 (64 GB/s) | Full PCIe 4.0 x16 (64 GB/s) |
| CPU | 32-core AMD EPYC | 32-core AMD EPYC | 2x AMD Genoa |
| System RAM | 128GB | 128GB | 384GB |
| System RAM Bandwidth | 204.8 GB/s | 204.8 GB/s | 921.6 GB/s |
| Disk Size | 4TB RAID Array + 1TB Boot | 4TB RAID Array + 1TB Boot | 1TB Boot |
| Disk Read Bandwidth | 28.7 GB/s | 28.7 GB/s | Use network |
| Networking | 2x 1GbE + open OCP3.0 slot (up to 200 GbE) | 2x 1GbE + open OCP3.0 slot (up to 200 GbE) | 2x open PCIe 5.0 x16 |
| Noise | Row 11 - Cell 1 | Row 11 - Cell 2 | Loud |
| Power Supply | 2x 1,600W, 100V to 240V | 2x 1,600W, 100V to 240V | 4x 2,000W, 200V+ |
| BMC | AST2500 | AST2500 | AST2600 |
| Operating System | Ubuntu 22.04 | Ubuntu 22.04 | Ubuntu 22.04 |
| Dimensions | 12U, 16.25” deep, 90 lbs. | 12U, 16.25” deep, 90 lbs. | 4U, 31” deep, 88 lbs. |
| Rack? | Freestanding or rack mount | Freestanding or rack mount | Supermicro rails |
| Driver Quality | Mediocre | Great | Great |
| Price | $15,000 | $25,000 | $40,000 |
Launching tinybox’s pro model is a good sign for the company, as it will likely see pent-up demand for lighter, more affordable AI accelerator alternatives to Nvidia and AMD. This is a major change as, just earlier this year, the tinybox project was put on hold due to issues with AMD’s drivers. It was eventually able to work with the company to fix the problems. However, the tinybox specifications still say that the driver quality for its tinybox red AI accelerator, which uses six 7900XTX GPUs, is ‘Mediocre.’
It is likely the primary reason tinygrad released the tinybox green, which uses six RTX 4090s. We’ve also heard it will release a tinybox blue model with Intel Arc GPUs, but it’s still in the prototype stage in August.
If you’re in the market for a powerful yet affordable AI accelerator, you can preorder the tinybox pro today. tinygrad asks for a 5% deposit ($2,000) to reserve your slot, and the order page says it will start shipping in December 2024.
