Related articles

• Tell me about CUDA, the "architecture," versus the "CUDA for C" compiler.CUDA is the name of Nvidia’s parallel computing hardware architecture. Nvidia provides a complete toolkit for programming the CUDA architecture that includes the compiler, debugger, profiler, libraries, and other information developers need to deliver production-quality products that use the CUDA architecture. The CUDA architecture also supports standard languages (C with CUDA extensions and Fortran with CUDA extensions) and APIs for GPU computing, such as OpenCL and DirectX Compute. This diagram may help: With OpenCL, you gain the advantage of cross-platform support, but lose automated tools, such as memory management, that are found with CUDA. It seems that as a scientist, you'd want to decrease your startup development costs, but at the same time, you'd want support for multiple platforms. What's the best way to reconcile this challenge? There are certainly compromises that have to be made to provide a cross vendor/platform solution. Nvidia has worked from the beginning with Apple and the OpenCL working group to make sure OpenCL provides a great driver-level API layer for GPU computing, especially for Nvidia hardware. Furthermore, we will certainly provide extensions to further enable Nvidia GPU’s with OpenCL. Nvidia is also constantly improving our C compiler and development environment for Nvidia GPUs. We have a few simple extensions to C in order to enable our GPUs. If Fortran is more your preference, there is a Fortran compiler also available. With the introduction of Microsoft’s Windows 7 this fall, users and developers will have access to the DirectCompute API, which shares many concepts with our C extensions. Nvidia seeded a DirectCompute driver to key developers last December. These are the added advantages to choosing Nvidia hardware; we support all major languages and API’s. Your work with GPUs started in the GeForce 5 era, and we're now several generations later. Obviously, the newer stuff is faster, but what new capabilities have been introduced over this time period (i.e. IEEE-754 compliance)? What can we do now that we couldn't do before? Early programmable GPUs were basic floating point-only programmable processors. No integer or bit operations, no general access to GPU memory, no communication between neighboring processors. The first main innovation was to provide the hardware needed for supporting C, which includes full pointer support and native data types. Another key innovation was the addition of dedicated, on-chip shared memory, which allows processors to intercommunicate and share results, greatly improving the efficiency of the algorithms. In addition, it offered programmers a place to temporarily store and process data close to the processor, rather than going all the way out to off-chip DRAMs. Shared memory improved our signal processing library by 20x over a similar OpenGL implementation. Finally the addition of double-precision floating point hardware also signified a key step toward GPUs as a true high performance computing product, enabling applications that required extended precision numerics. It should also be noted that memory speed and on-board memory size improvements (up to 4GB per processor and 16GB for our Tesla 1U server) has increased the scale of problems an Nvidia GPU can tackle. What about the compiler? What kind of optimizations and innovations have been added over time? Very early on, we recognized that we needed to build a world-class compiler solution. GPU computing programs tend to be much larger, more complex, and benefit from more complex optimization. Our competition (the CPU) had almost 40 years to get it right. Our C compiler is based on technologies from the Edison Design Group, who has been making C compilers for 20 years, and the Open64 compiler core, which was originally designed for the Itanium processor. Our compiler technology, combined with the world-class compiler team we’ve assembled, is a key part of Nvidia’s success. Currently, most GPUs are very fast with single precision math, but less quick with double precision math. Will GPUs still provide "better than CPU" cost/performance if it weren't for the economies of scale? That is, could you make a special double precision-optimized GPU while still keeping costs low? As the market for GPU computing clearly continues to grow, I think you will see more areas invested in double precision arithmetic. Our double precision hardware released last year was only the starting point for what I imagine will be a growing investment in GPU computing from both the industry and Nvidia. What is your impression of Intel's Larabee? AMD's Stream Architecture? Cell? Zii? My view of Larabee is that it is a great validation of what the GPU has achieved and an acceptance of the limitations the CPU. Where CPUs have tried to take a legacy sequential programming model and squeeze out every last bit of parallelism, GPUs were created for 3D rendering, an embarrassingly parallel application. Massive parallelism is a part of the GPU’s core programming model.  In the end, it is the accessibility and productivity of a programming model that will take an architecture from a novelty to a success. We are all competing against a mountain of legacy code. We’ve focused on making Nvidia GPUs extremely easy to obtain orders of magnitude speedups with a familiar and simple programming model.

• One of the interesting things about Larabee is the theoretical ability to do things like recursion on the chip. How would that compare to a theoretical approach of incorporating a lightweight x86 processor on a GPU for "housekeeping tasks?" I don’t think recursion is critical to the success of GPU computing, as almost all codes that run on the GPU are the performance-critical inner loops of an application. It is always best to inline and avoid things like recursions for performance reasons. We certainly could support recursion today, but prefer to allow our compiler to optimize without it. Regarding a lightweight CPU, there’s already a CPU in the system and we’ve focused on providing a razor-thin driver stack to keep things as efficient as possible. Where just-in-time processor scheduling is required, we’ve found dedicated hardware is almost always more area and power efficient at these critical tasks than a heavyweight x86 processor. Right now, the majority of GPGPU applications have been limited to scientific computing and video decoding/transcoding. Where do you see consumers benefiting from GPGPU technology in realms outside video?  The next wave of GPU computing consumer applications will be accelerating video editing, image processing, and gaming physics. We think your spreadsheet might already be fast enough. While video processing was an obvious application to accelerate, novel applications in computer vision, speech, and handwriting recognition applications for the consumer market can equally benefit from the massive performance potential of the GPU that is readily available in every PC. Where do you see GPGPU going in the future? Consumers are already benefiting from GPU computing. Companies like OptiTex are using CUDA to design clothing for the mass market. Car companies are designing next-generation cars with GPU ray tracing using CUDA. Physics engines in games are also migrating to the GPU. Moving forward, we’ll continue to see opportunities in personal media, such as sorting and searching photos based on the image content, i.e. faces, location, etc, is an incredibly compute-intensive operation. Some of the work I’m most proud of is in medical imagining and cancer research. Techniscan is a company using our Tesla GPUs to improve a doctor’s ability to detect and diagnose breast cancer earlier and more accurately than traditional methods. The National Cancer Institute is reporting a 12x CUDA-enabled speedup in protein ligand calculations used to design new drugs for diseases such as cancer and Alzheimer's. It is wonderful to see GPU computing being used in some of the fundamental research that will save lives.

• Exclusive Interview: Nvidia's Ian Buck Talks GPGPU

  Thanks for taking the time to chat today. Let's start with some basics. Why don't you tell our readers a little bit about yourself and what you currently do at Nvidia? I’m the Software Director for GPU Computing here at Nvidia. My main focus is to build and evolve a complete GPU computing platform, which includes system software, developer tools, language and compiler direction, libraries, and targeted applications and algorithms. With the help of a great team, we develop both the end-user software as well as set the direction for GPU computing within Nvidia. Why don't we start from the beginning? I imagine that your interest in GPGPU didn't start when you were 5 years old--what were the events going on at Princeton or Stanford that really led you to discover an interest in GPGPU? I did dabble in GPU computing in my Princeton days, experimenting with thermal convection and fluid simulation on graphics hardware, which, at the time was the SGI O2. Though things were so very constrained, it was hard to make a case for it. I seriously started looking at GPU computing during my PhD research at Stanford. At Stanford, I, along with others in the research community realized that the natural progression of programmable graphics was the evolution of the GPU into a more general purpose processor. We wrote one of the first SIGGRAPH papers on ray tracing with DX9-class GPU hardware to help prove the point. What was so motivating about the work was that this commodity processor, which was available in everyone’s PC, was following a Moore’s law cubed performance growth rate, way faster than the CPU. This begged the question: what could a PC do if it had multiple orders magnitude more computing horsepower than today? A total game-changer for the computational sciences as well as computer vision, AI, data mining, and graphics. What was your role with Brook? After working on the ability to ray trace on the GPU, my research focus at Stanford switched to understanding the right programming model for GPU computing. At the time, many others had shown that the GPU was good at a variety of different applications, but there wasn’t a good framework or programming model on how one should think about the GPU as a compute device. At the time, it required a PhD in computer graphics to be able to port an application to the GPU. So I started the Brook project with the goal of defining a programming language for GPU computing, which abstracted the graphics-isms of the GPU into more general programming concepts. Brook’s fundamental programming concept was the “stream,” which was a collection of data elements requiring similar work. Brook eventually became my PhD thesis at Stanford.  Your work started with Merrimac, the Stanford Streaming Super Computer. How is this different from something like a Tesla? Brook’s programming model concepts were applicable to more than just GPUs. At Stanford we worked on two different implementations of the Brook programming model: one for GPUs, the other for Merrimac which was a research architecture developed at Stanford. Many of the ideas pioneered as part of Merrimac did influence how GPUs could be improved for general purpose computing. It should also be noted that Bill Dally who was the principle investigator of Merrimac at Stanford, is now the Chief Scientist at Nvidia.  Did CUDA have any roots in Gelato? What was the first academic exploration of GPGPU? What about the first commercialized use? I started CUDA while completing my research at Stanford. Nvidia was already very supportive of my research and clearly saw the potential to better enable GPU computing on the hardware side of things. I joined Nvidia to start the CUDA project in 2005. At the time, it was just myself and one other engineer. We’ve now grown the project into the organization it is today, and a central component to Nvidia’s GPUs today. www.gpgpu.org provides a nice history of GPU computing, dating back to 2002. Currently, AMD pushes Brook as the programing language of choice for GPGPU, whereas Nvidia has C with CUDA extensions. How would you compare the strengths/weaknesses of both? Starting at Nvidia, we had an opportunity to revisit some of the fundamental design decisions of Brook, which were largely based on what DX9-class hardware could achieve. One of key limitations was the constraints of the memory model, which required the programmer to map their algorithm around a fairly limited memory access pattern. With our C with CUDA extensions, we relaxed those constraints. Fundamentally, the programmer was simply given a massive pool of threads and could access memory any way he or she wished. This improvement, as well as a few others, allowed us to implement full C language semantics on the GPU.