AI Startup Groq Debuts in the Cloud

Groq has become the second start-up to debut its deep learning chip in the cloud, EETimes reported. Its tensor streaming processor (TSP) for AI inference is now available via cloud service provider Nimbix for “selected customers”.

Late last year, Graphcore became the first startup to see its AI chip become available in the cloud through Microsoft’s Azure. Nimbix is now offering Groq’s tensor streaming processor.

Nimbix’ CEO stated: “Groq’s simplified processing architecture is unique, providing unprecedented, deterministic performance for compute intensive workloads, and is an exciting addition to our cloud-based AI and Deep Learning platform.”

Groq’s TSP is rated at 1000 TOPS (1POPS) and the company claims it has 2.5x the performance of the best GPUs at a large batch size, and its performance lead would be 17x at a batch size of 1.

Besides GPUs, the chip competes with many other deep learning inference chips. Most notably, Intel and Qualcomm will also seek to get their own NNP-I and Cloud 100 chips in the cloud this year.

WikiChip has a deep dive on the chip.