TinyBox packs a punch with six of AMD's fastest gaming GPUs repurposed for AI — new box uses Radeon 7900 XTX and retails for $15K, now in production

AMD
(Image credit: AMD)

Tiny Corp., the startup that maintains tinygrad, a neural network framework, aims to democratize PetaFLOPS-class performance for artificial intelligence. To that effect, it has unveiled its TinyBox system that packs six AMD's Radeon RX 7900 XTX graphics cards into a 12U rack case and will sell for $15,000. That figure includes all the necessary software and hardware.

The TinyBox system uses six AMD Radeon RX 7900 XTX boards (one of the best graphics cards) connected using 'full fabric' PCIe 4.0 x16 links to ensure maximum bandwidth. Apparently, these consumer-grade GPUs fully support the peer-to-peer interconnections necessary for large language models (unlike Nvidia's GeForce RTX 4090), so the company chose to use these GPUs over Nvidia's more popular options. 

From a performance point of view, the TinyBox can offer up to 738 FP16 TFLOPS (0.738 FP16 TFLOPS) of performance along with 96 GB of GDDR6 memory that delivers 21 TB/s of aggregated memory bandwidth. To put the numbers into context, one TinyBox offers 37% of Nvidia H100 compute performance (FP16) but slightly more memory (96 GB instead of 80 GB) and considerably higher peak memory bandwidth (21 TB/s instead of 3.35 TB/s). 

The system is powered by an AMD EPYC 7532 CPU mated with 128 GB of RAM. As for storage, the TinyBox has five Western Digital SN850X 1TB SSDs, four configured in RAID for enhanced performance and one dedicated for booting the system. Additionally, the machine features an empty 16x OCP 3.0 slot for networking, allowing for flexible connectivity options. 

(Image credit: Tiny)

Tiny says that the cost of its TinyBox machine is about $10,000, and it sells it for $15,000, which is considerably lower than Nvidia's H100. As a result, the company has received 583 preorders and plans to begin shipping in April. A production run of 100 units is already underway, the company indicated. 

On the software side, the TinyBox ships with Ubuntu 22.04 and initially will include only the tinygrad framework. However, it is compatible with other machine learning frameworks such as PyTorch and JAX on AMD hardware, providing users with the flexibility to choose their preferred tools.  

Since Tiny wants to democratize PetaFLOPS-class performance for AI workloads, the TinyBox is not just a product, but rather a testament that this can be done. By providing a standardized piece of hardware, the company aims to enable developers to explore new frontiers in software and algorithm development, ultimately leading to breakthroughs in the field. As AMD, Intel, and Nvidia introduce new high-performance compute architectures based on data center-grade hardware, we can only wonder whether TinyBox, based on consumer-grade hardware, will gain popularity.

Anton Shilov
Contributing Writer

Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.

  • Notton
    oh cool, yet another startup that will cause gaming GPU prices to spike.
    Reply
  • ace6558966
    And the award for least density per ru goes to.... but seriously, why in the world does this solution necessitate 12u (expansion plans???)? 6gpu, an epyc Rome CPU,128gb of ram,5 SSDs, ocp port, and what?
    Reply
  • HaninTH
    I need one of these to run my Home Assistant server!
    Reply
  • Order 66
    Too bad crossfire is dead. Could you imagine the power of 6 7900xtx's? (assuming of course the game was actually coded for it)
    Reply
  • digitalgriffin
    The large memory models will make it a good choice for training and image generation. But the poor compute performance will make it bad with inference and the likes where the target vector is compared against inputs.
    Reply
  • USAFRet
    Order 66 said:
    Too bad crossfire is dead. Could you imagine the power of 6 7900xtx's? (assuming of course the game was actually coded for it)
    You couldn't afford to buy the game.
    Reply
  • Order 66
    USAFRet said:
    You couldn't afford to buy the game.
    Very true. 6 7900xtx's as well as a CPU capable of running them all isn't cheap after all. A man can dream, though
    Reply
  • USAFRet
    Order 66 said:
    Very true. 6 7900xtx's as well as a CPU capable of running them all isn't cheap after all. A man can dream, though
    I meant just the game code.

    To take advantage of that hardware, it would need a whole new custom version.

    There would be a global market of maybe 25 people.

    Salaries of a dozen people on the dev team, divided by 25...
    Reply
  • Order 66
    USAFRet said:
    I meant just the game code.

    To take advantage of that hardware, it would need a whole new custom version.

    There would be a global market of maybe 25 people.

    Salaries of a dozen people on the dev team, divided by 25...
    Yeah, that's a great point. Maybe more than 25 people, though. I mean there are thousands of people that can afford a shiny new flagship GPU for $1000+ every 2 years. Who's to say that those same people couldn't save up their money for a few years and buy the hardware as well as the game? Is it likely or even feasible, no of course not, but it could happen.
    Reply
  • USAFRet
    Order 66 said:
    ...but it could happen.
    And monkeys could fly outta my butt.
    Reply