The CUDA APIs
But, Brook’s critical success was enough to attract the attention of ATI and Nvidia, since the two giants saw the incipient interest in this type of initiative as an opportunity to broaden their market even more by reaching a new sector that had so far been indifferent to their graphics achievements.
Researchers who were in on Brook at its origin quickly joined the Santa Clara development teams to put together a global strategy for targeting the new market. The idea was to offer a hardware/software ensemble suited to this type of calculation – since Nvidia’s developers know all the secrets of their GPU, there was no question of relying only on a graphics API, which only communicates with the hardware via a driver, with all the problems that implies, as we saw above. So the CUDA (Compute Unified Device Architecture) development team created a set of software layers to communicate with the GPU.
As you can see on this diagram, CUDA provides two APIs:
- A high-level API: the CUDA Runtime API;
- A low-level API: the CUDA Driver API.
Since the high-level API is implemented “above” the low-level API, each call to a function of the Runtime is broken down into more basic instructions managed by the Driver API. Note that these two APIs are mutually exclusive – the programmer must use one or the other, but it’s not possible to mix function calls from both. The term “high-level API” is relative. Even the Runtime API is still what a lot of people would consider very low-level; yet it still offers functions that are highly practical for initialization or context management. But don’t expect a lot more abstraction – you still need a good knowledge of Nvidia GPUs and how they work.
The Driver API, then, is more complex to manage; it requires more work to launch processing on the GPU. But the upside is that it’s more flexible, giving the programmer who wants it additional control. The two APIs are capable of communicating with OpenGL or Direct3D resources (only nine for the moment). The usefulness of this is obvious – CUDA could be used to generate resources (geometry, procedural textures, etc.) that could then be passed to the graphics API, or conversely, it’s possible that the 3D API could send the results of the rendering to CUDA, which in that case would be used to perform post-processing. There are numerous examples of interactions, and the advantage is that the resources remain stored in the GPU’s RAM without having to transit through the bottleneck of the PCI-Express bus.
Conversely, we should point out that sharing resources – in this case video memory – with graphics data is not always idyllic and can lead to a few headaches. For example, for a change of resolution or color depth, the graphics data have priority. So, if the resources for the frame buffer need to increase, the driver won’t hesitate to grab the ones that are allocated to applications using CUDA, causing them to crash. It’s not very elegant, granted; but you have to admit that the situation shouldn’t come up very often. And since we’re on the subject of little disadvantages: If you want to use several GPUs for a CUDA application, you’ll have to disable SLI mode first, or only a single GPU will be visible to CUDA.
Finally, the third software layer is a set of libraries – two to be precise:
- CUBLAS, which has a set of building blocks for linear algebra calculations on the GPU;
- CUFFT, which can handle calculation of Fourier transforms – an algorithm much used in the field of signal processing.
Current page: The CUDA APIsPrev Page BrookGPU Next Page A Few Definitions
Stay on the Cutting Edge
Join the experts who read Tom's Hardware for the inside track on enthusiast PC tech news — and have for over 25 years. We'll send breaking news and in-depth reviews of CPUs, GPUs, AI, maker hardware and more straight to your inbox.
CUDA software enables GPUs to do tasks normally reserved for CPUs. We look at how it works and its real and potential performance advantages.Reply
Nvidia's CUDA: The End of the CPU? : Read more
Well if the technology was used just to play games yes, it would be crap tech, spending billions just so we can play quake doesnt make much sense ;)Reply
Wow a gaming GFX into a serious work horse LMAO.Reply
The Best thing that could happen is for M$ to release an API similar to DirextX for developers. That way both ATI and NVidia can support the API.Reply
And no mention of OpenCL? I guess there's not a lot of details about it yet, but I find it surprising that you look to M$ for a unified API (who have no plans to do so that we know of), when Apple has already announced that they'll be releasing one next year. (unless I've totally misunderstood things...)Reply
Im not gonna bother reading this article, I just thought the title was funny seeing as how Nvidia claims CUDA in NO way replaces the CPU and that is simply not their goal.Reply
I´d like it better if DirectX wouldnt be used.Reply
Anyways, NV wants to sell cuda, so why would they change to DX ,-)
I think the best way to go for MS is announce to support OpenCL like Apple. That way it will make things a lot easier for the developers and it makes MS look good to support the oen standard.Reply
Mr RobotoVery interesting. I'm anxiously awaiting the RapiHD video encoder. Everyone knows how long it takes to encode a standard definition video, let alone an HD or multiple HD videos. If a 10x speedup can materialize from the CUDA API, lets just say it's more than welcome.I understand from the launch if the GTX280 and GTX260 that Nvidia has a broader outlook for the use of these GPU's. However I don't buy it fully especially when they cost so much to manufacture and use so much power. The GTX http://en.wikipedia.org/wiki/Gore-Tex 280 has been reported as using upwards of 300w. That doesn't translate to that much money in electrical bills over a span of a year but never the less it's still moving backwards. Also don't expect the GTX series to come down in price anytime soon. The 8800GTX and it's 384 Bit bus is a prime example of how much these devices cost to make. Unless CUDA becomes standardized it's just another niche product fighting against other niche products from ATI and Intel.On the other hand though, I was reading on Anand Tech that Nvidia is sticking 4 of these cards (each with 4GB RAM) in a 1U formfactor using CUDA to create ultra cheap Super Computers. For the scientific community this may be just what they're looking for. Maybe I was misled into believing that these cards were for gaming and anything else would be an added benefit. With the price and power consumption this makes much more sense now. Agreed. Also I predict in a few years we will have a Linux distro that will run mostly on a GPU.Reply