Related articles

• The two worlds remained separate for a long time. We used the CPU (or several CPUs) for office and Internet applications and GPUs were good only for drawing pretty pictures faster. But a single event would change all that: the appearance of programmability in GPUs. At first, CPUs had nothing to fear. The first so-called programmable GPUs (the NV20 and R200) were far from being a threat. The number of instructions for a program remained limited to around 10, and they worked on exotic data types like nine- or 12-bit fixed-point numbers. But Moore’s Law rears its head once again. Not only does the increase in the number of transistors make it possible to increase the number of calculating units, but it also increases their flexibility. So, the appearance of the NV30 was significant for several reasons. While gamers may not induct the NV30 into their hall of fame, it did usher in two factors that were important in changing the mindset that sees GPUs as nothing more than graphics accelerators: support for single-precision floating-point calculations (even if it didn’t comply with the IEEE754 standard);support for a number of instructions in excess of a thousand. At this point, all the conditions were in place to attract a few curious researchers on the lookout for ways to wring out more processing power.

• Nvidia introduced CUDA with the release of the GeForce 8800. At that time the promises they were making were extremely seductive, but we kept our enthusiasm in check. After all, wasn’t this likely to be just a way of staking out the territory and surfing the GPGPU wave? Without an SDK available, you can’t blame us for thinking it was all just a marketing operation and that nothing really concrete would come of it. It wouldn’t be the first time a good initiative has been announced too early and never really saw the light of day due to a lack of resources – especially in such a competitive sector. Now, a year and a half after the announcement, we can say that Nvidia has kept its word. Not only was the SDK available quickly in a beta version, in early 2007, but it’s also been updated frequently, proving the importance of this project for Nvidia. Today CUDA has developed nicely; the SDK is available in a beta 2.0 version for the major operating systems (Windows XP and Vista and Linux and 1.1 for Mac OS X), and Nvidia is devoting an entire section of its site for developers. On a more personal level, the impression we got from our first steps with CUDA was extremely positive. Even if you’re familiar with the GPU’s architecture, it’s natural to be apprehensive about programming it, and while the API looks clear at first glance you can’t keep from thinking it won’t be easy to get convincing results with the architecture. Won’t the gain in processing time be siphoned off by the multiple CPU-GPU transfers? And how to make good use of those thousands of threads with almost no synchronization primitive? We started our experimentation with all these uncertainties in mind. But they soon evaporated when the first version of our algorithm, trivial as it was, already proved to be significantly faster than the CPU implementation. So, CUDA is not a gimmick intended for researchers who want to cajole their university into buying them a GeForce. CUDA is genuinely usable by any programmer who knows C, provided he or she is ready to make a small investment of time and effort to adapt to this new programming paradigm. That effort won’t be wasted provided your algorithms lend themselves to parallelization. We should also tip our hat to Nvidia for providing ample, quality documentation to answer all the questions of beginning programmers. For the latest on CUDA click here.

• However we did decide to measure the processing time to see if there was any advantage to using CUDA even with our crude implementation, or on the other hand if was going to take long, exhaustive practice to get any real control over the use of the GPU. The test machine was our development box – a laptop computer with a Core 2 Duo T5450 and a GeForce 8600M GT, operating under Vista. It’s far from being a supercomputer, but the results are interesting since our test is not all that favorable to the GPU. It’s fine for Nvidia to show us huge accelerations on systems equipped with monster GPUs and enormous bandwidth, but in practice many of the 70 million CUDA GPUs existing on current PCs are much less powerful, and so our test is quite germane. The results we got are as follows for processing a 2048x2048 image: CPU 1 thread: 1419 msCPU 2 threads: 749 msCPU 4 threads: 593 ms GPU (8600M GT) blocks of 256 pixels: 109 msGPU (8600M GT) blocks of 128 pixels: 94 msGPU (8800 GTX) blocks of 128 pixels / 256 pixels: 31 ms Several observations can be made about these results. First of all you’ll notice that despite our crack about programmers’ laziness, we did modify the initial CPU implementation by threading it. As we said, the code is ideal for this situation – all you do is break down the initial image into as many zones as there are threads. Note that we got an almost linear acceleration going from one to two threads on our dual-core CPU, which shows the strongly parallel nature of our test program. Fairly unexpectedly, the four-thread version proved faster, whereas we were expecting to see no difference at all on our processor, or even – and more logically – a slight loss of efficiency due to the additional cost generated by the creation of the additional threads. What explains that result? It’s hard to say, but it may be that the Windows thread scheduler has something to do with it; but in any case the result was reproducible. With a texture with smaller dimensions (512x512), the gain achieved by threading was a lot less marked (approximately 35% as opposed to 100%) and the behavior of the four-thread version was more logical, showing no gain over the two-thread version. The GPU was still faster, but less markedly so (the 8600M GT was three times faster than the two-thread version). The second notable observation is that even the slowest GPU implementation was nearly six times faster than the best-performing CPU version. For a first program and a trivial version of the algorithm, that’s very encouraging. Notice also that we got significantly better results using smaller blocks, whereas intuitively you might think that the reverse would be true. The explanation is simple – our program uses 14 registers per thread, and with 256-thread blocks it would need 3,584 registers per block, and to saturate a multiprocessor it takes 768 threads, as we saw. In our case, that’s three blocks or 10,572 registers. But a multiprocessor has only 8,192 registers, so it can only keep two blocks active. Conversely, with blocks of 128 pixels, we need 1,792 registers per block; 8,192 divided by 1,792 and rounded to the nearest integer works out to four blocks being processed. In practice, the number of threads are the same (512 per multiprocessor, whereas theoretically it takes 768 to saturate it), but having more blocks gives the GPU additional flexibility with memory access – when an operation with a long latency is executed, it can launch execution of the instructions on another block while waiting for the results to be available. Four blocks would certainly mask the latency better, especially since our program makes several memory accesses.