Multiple Anti-Aliasing Modes For Multiple Scenarios
We covered a lot of performance data encompassing multiple anti-aliasing modes, and the implementations are so different that we can't draw a single conclusion about them all. Instead, we’ll have to consider each mode separately:
Multi-Sampling Anti-Aliasing (MSAA)
Although MSAA doesn’t address all manifestations of aliasing, it does fix the most blatant problems. Of the basic 2x, 4x, and 8x MSAA modes, we'd have to say that 4x offers the best tradeoff between quality and performance. Stepping down to 2x MSAA sacrifices image quality without boosting performance much, while 8x MSAA can cause a big frame rate hit without a proportionate quality increase.
Mid-range graphics cards like the GeForce GTX 460 and Radeon HD 6850 can handle 4x MSAA in most game engines and still maintain playable performance. Even the GeForce GTX 550 Ti and Radeon HD 5770 manage 4x MSAA in some games.
Lower-end graphics hardware will likely force you to choose between anti-aliasing and lower detail settings in order to maintain suitable frame rates. That decision often boils down to personal preference.
The highest-quality AA mode, supersampling essentially renders the frame at a higher resolution and downsamples the result. This causes a performance hit so large that Nvidia straight-up removed this mode from its GeForce drivers some time ago.
AMD left the supersampling option in its Catalyst driver, however, and while it doesn’t work in most games, it’s very attractive. Just be sure you have powerful graphics hardware (like a Radeon HD 6970 or high-end CrossFire setup) if you want playable frame rates. Even with the right stuff under the hood, 8x supersampling AA slows performance to a crawl, so it’s best to limit this fringe mode to the 4x setting.
Coverage Sample Anti-Aliasing (CSAA for GeForce, EQAA for Radeon)
This mode offers a slight improvement over standard MSAA, adding coverage samples to multi-samples. Unfortunately, any quality improvement enabled by coverage samples pales in comparison to using multi-samples. Having said that, the performance hit of coverage sampling is very small, so choosing 4x multisample plus 4x coverage sample modes like Nvidia’s 8x CSAA or AMD’s 4xEQAA isn’t a bad way to realize a little extra visual quality. CSAA works across the GeForce line, but EQAA is limited to Radeon HD 6900-series cards.
This Radeon-exclusive mode uses an edge-detect algorithm combined with the ability to sample outside the pixel boundary. Success varies based on the scene and game, but the result usually subtle like the improvements typical of coverage sampling anti-aliasing. Unfortunately, edge-detect AA causes more of a performance hit than higher levels of MSAA. For example, 12x edge-detect AA is clearly inferior to 8x MSAA, despite lower frame rates. Because of this, we don’t recommend edge-detect AA unless you can get frame rates that are high enough to make 24x edge-detect AA viable without sacrificing playability.
Texture Transparency Anti-Aliasing (TrAA for GeForce, Adaptive AA for Radeon)
Largely ignored, but still capable of delivering significant quality improvements where transparent textures are used, these modes can work wonders. Unfortunately, Nvidia’s multi-sampling TrAA and AMD’s Adaptive AA very rarely cooperate with modern game engines. The only mode that is somewhat reliable is Nvidia’s supersampling TrAA, which thankfully exacts a relatively small performance hit.
Because of its benefits and relatively low impact on performance, we recommend forcing 2x or 4x supersampling TrAA, depending on what your graphics hardware can handle. It can be configured independently of MSAA, so supersampling TrAA can be set and forgotten.
Another Radeon-exclusive mode, morphological anti-aliasing is a post-process filtering technique powered by pixel shaders. It does an impressive job of minimizing aliasing artifacts on the entire scene, but can adversely impact the quality of a scene's interface and text elements. This mode incurs a substantial frame rate hit that increases with resolution. On the plus side, MLAA can be forced on when other anti-aliasing modes may not be available. So, despite its hefty performance penalty, it's a good tool to have available in limited situations. You’ll probably want at least a Radeon HD 6850 in order to use it, though.