Vertex Shaders and Pixel Shaders

Fragment Vertex Processing

Fragment processing is an idea that's come to the foreground since DX8 was released to game developers. Appreciating that the number of possible shaders can get ridiculously large, nVidia have introduced the NVLink tool, which allows us to build shaders dynamically, from a set of "fragments."

This approach has a number of good sides, and a fair number of bad sides, too. To start with, we need to define the fragment architecture that we wish to place over the top of the vertex shader architecture. Seeing as the main thing we want to do with this architecture is to emulate elements of the fixed-function pipeline, it's probably best to start with something resembling that architecture.

A simple fragment based vertex processor.

For a typical game engine, it's easy to see where we'll want to carry out each type of vertex processing. Our split between skinned vertices, untransformed vertices and transformed vertices occurs within the transformation block of the fragment processor.

Different lighting models are catered for by the lighting block, although there may be some interaction with the material block if we are using bump-mapping, which will typically take over the processing of light data, and effectively switch per-vertex lighting off.

The texture and material block takes care of various material models. The values it can affect are the diffuse, specular registers (for bump-mapping), and the texture coordinate output registers.

The lighting block of the pipeline will want to have access to the position and normal of the vertex. To maintain flexibility, it may be necessary for the texture and material pipeline to know about the position and normal of a vertex (useful for various effects), so our final block is called projection, and moves our world space position from a temporary register where it can be accessed by the material and lighting blocks, through a projective transform, and into the output register (which is write-only).

For each fragment to work together, the programmers using the system will have to adopt some basic programming conventions. Each fragment will need to know where the data from previous fragments has been placed. This can be done by stating that certain temporary registers will contain certain vertex data, or by using identifiers and letting the linker resolve names to handle register renaming. The latter case is already supported in NVLink, although, for in-house systems, it will be easier to establish temporary register conventions.

The introduction of processing blocks also introduces the need for automated optimization of the vertex shaders. It's easy to see that the transformation block might move the world space position into a temporary register, even though the texture and material block doesn't need this data. The output position block then carries out a wasted move instruction. It would have been more sensible to place the position directly in the output register during the transformation fragment. Linkers can be made to spot this type of optimization, along with the more difficult case where a multiply and add instruction could be pulled into a single, mad instruction.

Once the necessary architecture for fragment processing is in place, there are a number of interesting avenues open to the game developer. The most powerful possibility is to allow global distortion effects to be added to all of the vertex shaders in the world. To do this, we can add a new fragment block after the transformation block to carry out arbitrary world space distortion. Then, when we want to add the effect to the world, we can insert the new fragment into all of the existing vertex shaders and re-link them. Now all of the geometry in the world can wave in the wind, or be distorted from the center of a shockwave using a cosine wave, or whatever you can imagine.

This level of involvement is probably overkill for most game engines, but it's possible that the extra architecture will be necessary for graphics engines that are attempting to be at the "bleeding edge" of technology.