The major new feature of Direct3D 10 was the appearance of Geometry Shaders, which finally made it possible to create or destroy vertices on the GPU. But the role of this unit has been somewhat misinterpreted. Rather than being used for massive expansion of geometry, it’s better suited to implementing more flexible Point Sprites, managing Fur Shading, or calculating the silhouette of an object for shadow volume algorithms. Nothing is better than a dedicated unit for doing tessellation. Initially planned for Direct3D 10 (which explains its presence in the Radeon HD series), it seems that Microsoft, ATI, and Nvidia weren’t able to reach an agreement in time, and so it disappeared from the specifications, only to return with a vengeance with Direct3D 11. So, tessellation is the big new feature of Direct3D 11—or at least the one that’s easiest to sell for non-specialists.
So, Microsoft has introduced three new stages in its rendering pipeline:
Unlike other stages of the pipeline, these don’t operate with triangles as primitives, but use patches. The Hull Shader takes the control points for a patch as input and determines certain parameters of the Tesselator, such as, for example, TessFactor, which indicates the degree of fineness of the tessellation. The Tesselator is a fixed-function unit, so the programmer does not control how tessellation is calculated. The unit sends the points generated to the Domain Shader, which can apply operations to them. An example should make all this clearer. Let’s take a case that has come up in each generation since Matrox’s Parhelia—Displacement Mapping.
As input to the vertex shader, we have the control points of the patch. The programmer can manipulate these as he or she sees fit, since they aren’t numerous. To simplify, they are a very coarse version of the final mesh. These transformed points are then passed to the Hull Shader, which determines how many times to subdivide each side of the patch (for example, as a function of the size of the patch in pixels on the display). The Tesselator handles tessellation as such. That is, the creation of the geometry, which is passed to the Domain Shader. It transforms the points generated into the appropriate space (the points exiting the Tesselator are in the space of the patch), producing classic vertices that it can displace as a function of a texture and perform Displacement Mapping.
The potential is enormous. Thanks to tessellation, it becomes possible to do without the normal map and implement a level of detail directly on the GPU, allowing the use of very detailed models (several million polygons instead of 10,000 or so with current games)—at least in theory. In practice, tessellation raises some problems that have kept the technique from taking off so far. Can Direct3D 11 and compatible cards avoid these pitfalls and come up with a functional version? It’s too early to say, but in any event not everybody is convinced, and id Software is working on solving the same geometry problem with a completely different approach based on ray casting with voxels.