There are loads of compact modern workstations that pack quite capable CPUs, but at the same time they lack the space to accommodate a standard high-performance workstation-grade graphics card. That typically limits them to entry-level GPUs with mediocre performance. For those that want a compact SFF workstation with more graphics oomph, Nvidia has introduced a new ProViz-oriented RTX 4000 SFF Ada Generation graphics card. It's one of the more interesting offerings in the recent years, packing a high-end GPU into a low-profile form-factor, with a power consumption of just 70W.
The Nvidia RTX 4000 SFF Ada board uses the company's AD104 graphics processing unit with 6144 CUDA cores enabled (out of 7680 in total). That's the same GPU as the RTX 4070 Ti but with fewer active cores, and the boost frequency gets capped at around 1560 MHz to lower total board power. On the other hand, the graphics card comes with 20GB of GDDR6 memory with ECC that connects to the GPU using a 160-bit interface, so lots of memory for workstation use.
The GPU comes with two NVENC encoders and two NVDEC decoders activated, though Nvidia has not touched upon exact capabilities of these units. They should be similar to the NVENC and NVDEC used in other Ada cards, and you can see the video encoding performance and quality in our recent roundup of GPUs.
The GA104 chip in this configuration delivers peak single precision performance of 19.2 TFLOPS, making it theoretically comparable to a GeForce RTX 3070. It has peak RT performance of 44.3 TFLOPS, and peak FP8/INT8 tensor performance of 306.8 TFLOPS/TOPS.
Almost 20 FP32 TFLOPS may be dwarfed by the overwhelming performance of Nvidia’s RTX 6000 Ada Generation or GeForce RTX 4090, but the RTX 4000 SFF is a low-profile dual-slot graphics card that can fit into almost any desktop computer, even one that does not have a spare auxiliary PCIe power connector. Interestingly, RTX 4000 Ada’s 153/306.8 INT8 TFLOPS (without and with sparsity, respectively) performance is very close to that of Nvidia’s GeForce RTX 3090 Ti that is both more expensive and far more power hungry.
|Row 0 - Cell 0||GPU||FP32 CUDA Cores||FP32 TFLOPS||INT8 TFLOPS||Memory Configuration||TBP||MSRP|
|GeForce RTX 4070 Ti||AD104||7680||40 TFLOPS||160/320 TFLOPS||12GB 192-bit 21 GT/s GDDR6X||285W||$799|
|GeForce RTX 4070||AD104||5888 (?)||?||?||12GB 192-bit 21 GT/s GDDR6X||250W (?)||?|
|RTX 4000 Ada Generation||AD104||6144||19.2 TFLOPS||153/307 TFLOPS||20GB 160-bit 16 GT/s GDDR6 ECC||70W||$1,250|
|GeForce RTX 3090 Ti||GA102||10,752||40 TFLOPS||160/320 TFLOPS||24GB 384-bit 20 GT/s GDDR6X||450W||$1,999|
|GeForce RTX 3070||GA104||5888||20.31 TFLOPS||81/160 TFLOPS||8GB 256-bit 14 GT/s GDDR6||220W||$499|
Since this is a workstation-grade add-in-board, it comes with four DisplayPort 1.4a connectors, has a 3-pin mini-DIN connector for stereoscopic 3D output (e.g. Nvidia 3D Vision), and supports Frame Lock capability for multi-display applications.
Speaking of multi display applications, one of the benefits of the compact dimensions, low power consumption, and broad compatibility of Nvidia’s RTX 4000 Ada Generation graphics card is the ability to install a number of such boards into a relatively compact system. It wouldn't need a high-wattage PSU and could still drive multi-display and video wall applications. Such systems are widely used by various industries, including aerospace, healthcare, military, pro A/V, digital signage, and security, just to name a few.
Starting in April, the newly released Nvidia's RTX 4000 SFF graphics cards for professional visualization applications will be available from the company's distribution partners like Leadtek, PNY, and Ryoyo Electro, with a recommended price of $1,250. Additionally, workstation manufacturers will offer this product later in the year.
Not sure why you would compare it with a 3090Ti tech should get better and cheaper. Not better and more expensive else it is not progress.
RTX A2000 is popular card because it is basically an RTX2060 that operates also 70W and is low profile capable.
If this one sells poorly, maybe it will also come down in price and become the new SFF card of choice.
Its a workstation card, Quadros are always more expensive...certified drivers are very expensive.
It has ECC memory, which makes it a Quadro without the name "quadro"
Your work should get better and your wages lower or it's not progress. Sound stupid? Yeah, it does. Just like your comment.
More productivity should come at a cost. And for the end user, 300w vs 70w (times countless employees) will payoff that GPU cost over time. This isn't some low-wage-earning couch potato gamer card.
Imagine me penalizing your paycheck because you did 8hrs of work in 4 hrs. What do you value more? Time, or your money? Anyone wanting to be productive values time over money. Those who can work faster deserve more money, not less. Working faster than yesterday IS progress.
Most NLP workstation loads (the most demanding of AI), including GPT-4 training, can be done under 10GB and 12GB. That also applies to stable diffusion too.
This isn't an enterprise card designed for production-class workloads on a server doing training and/or nference for thousands of users at a time.
Also, most ML/AI training and inference (with the exception of NLP) can still be done on 12GB (x2) K80, albeit slower. Heck, a whole lot of ML work can still be done on a 1080ti. Ask me how I know.
Not a Kaggle user, are you?
At least someone around here knows and understands this isn't a game's play toy. I don't even think the writer of this article even does. On the plus side, at least they didn't compare it to any AMD cards like unproductive couch potato gamers have.