# ==Real-time anti-aliasing==, MSAA / FXAA / SMAA / TAA <p class="doc-sub">// status: seedling</p> Aliasing is undersampling: a high-frequency edge, texture, shadow, or shader detail changes faster than the pixel grid can represent it. Anti-aliasing is not one algorithm because the renderer can sample different things at different stages. [[Deferred vs forward rendering|Forward]] and deferred pipelines also give those samples different costs. The practical question is not “which AA is best?” It is “which aliasing do I have, and where can I afford to sample it?” ## The short comparison | Technique | Samples | Fixes well | Does not fix well | Cost / memory | |---|---|---|---|---| | **MSAA** | multiple coverage/depth samples per pixel | geometric triangle edges | shader, texture, shadow, and post-process aliasing | raster + multisample attachments | | **FXAA** | one rendered image, local edge filter | obvious contrast edges, very cheap | subpixel detail, motion, text sharpness | one post-process pass | | **SMAA** | image edges + pattern/search tables | thin lines and diagonal edges better than FXAA | temporal shimmer, information already lost | 2–3 post-process passes | | **TAA** | jittered frames + reprojected history | geometry, shader, foliage, and subpixel detail over time | disocclusion, ghosting, unstable motion vectors | history + velocity + resolve pass | There is a hybrid option: MSAA for coverage plus TAA for temporal and shader aliasing. It costs more bandwidth, but can be worthwhile in a forward renderer with expensive foliage or alpha-tested geometry. ## MSAA: sample coverage where rasterization knows it With multisample anti-aliasing, a pixel has several coverage/depth samples. A triangle may cover two of four samples, so the final resolve mixes its colour with the background according to coverage. The fragment shader can run once per pixel for a uniform result, or per sample when a shader actually needs sample-frequency values. Conceptually: ```text for each pixel: coverageMask = rasterizerCoverage(triangle, sampleLocations) depthTest each covered sample shade once (or per sample when required) resolve = average surviving sample colours ``` OpenGL creates multisample textures/renderbuffers with a sample count and uses the multisample framebuffer state; Vulkan puts the sample count in the attachment/pipeline state and can resolve into a single-sample attachment. Both APIs require compatible sample counts for the attachments in a render pass or dynamic-rendering instance. MSAA's strengths are also its limits: - It sees triangle coverage before the fragment shader, so it gives clean polygon silhouettes. - It does not automatically supersample a procedural shader, a shadow lookup, a normal map, or a post-processing edge. - Alpha-tested foliage needs careful coverage/sample-mask handling; alpha blending is not magically solved. - A deferred G-buffer must either store per-sample attributes or shade a resolved/alternative representation. That is why deferred pipelines commonly combine a lower sample count with TAA. Resolve in linear colour space. Resolving encoded sRGB values as if they were linear makes edges too dark or too bright. Let the framebuffer format and API perform the correct conversion, or make the decode/resolve order explicit. ## FXAA: a useful last-resort filter Fast Approximate Anti-Aliasing is a screen-space pass. It detects a local luminance contrast, estimates the edge direction, samples along that direction, and blends toward a result that crosses the edge more smoothly. A deliberately simplified version looks like: ```glsl float luma(vec3 c) { return dot(c, vec3(0.299, 0.587, 0.114)); } vec3 fxaa(vec2 uv, vec2 texelSize) { vec3 cM = texture(scene, uv).rgb; float lM = luma(cM); float lN = luma(texture(scene, uv + vec2(0, 1) * texelSize).rgb); float lS = luma(texture(scene, uv - vec2(0, 1) * texelSize).rgb); float lE = luma(texture(scene, uv + vec2(1, 0) * texelSize).rgb); float lW = luma(texture(scene, uv - vec2(1, 0) * texelSize).rgb); float contrast = max(max(lN, lS), max(lE, lW)) - min(min(lN, lS), min(lE, lW)); if (contrast < edgeThreshold) return cM; vec2 direction = normalize(vec2(lN - lS, lW - lE)); vec3 a = texture(scene, uv + direction * texelSize * 0.5).rgb; vec3 b = texture(scene, uv - direction * texelSize * 0.5).rgb; return mix(cM, 0.5 * (a + b), edgeWeight); } ``` Production FXAA uses more carefully tuned directional searches, subpixel controls, and luma range reduction; use the implementation and quality settings from the original reference rather than treating this sketch as a drop-in shader. FXAA is attractive for a simple [[OpenGL - learning log|OpenGL]] forward renderer because it needs no multisample attachments and no history. The trade is a generally softer image and no temporal stability: a thin line can flicker as it moves one pixel at a time. ## SMAA: reconstructing edge shapes Subpixel Morphological Anti-Aliasing keeps the same post-process placement but uses a more structured edge search: 1. **Edge detection** finds horizontal and vertical discontinuities. 2. **Pattern / search** textures estimate how a diagonal or corner is covered. 3. **Blend weights** combine neighbouring pixels according to that inferred shape. The original SMAA implementation also describes temporal modes that feed a history of edge information. The basic spatial version is a strong choice when FXAA is too soft and TAA is undesirable (for example, a static CAD view or a UI-heavy application). It costs multiple passes and the lookup textures add setup, but its thin-edge behaviour is usually much better than a simple blur. SMAA and FXAA both operate after the scene has already been sampled. They cannot restore texture detail that was minified incorrectly or reconstruct a surface hidden by a previous shader branch. Use proper mipmapping and anisotropic filtering first; post-AA is not a replacement for sampling the material correctly. ## TAA: spend samples over time Temporal anti-aliasing jitters the projection by a subpixel offset each frame, then reprojects the previous result into the current frame. A stable pixel accumulates different sample positions over time. The hard part is deciding when history is still valid. The minimum useful inputs are: - current colour, preferably in HDR linear space; - previous history colour; - motion vectors from current to previous frame; - depth, and often a normal or material ID for rejection; - the exact current and previous jittered camera matrices. ```glsl vec3 resolveTaa(vec2 uv, vec2 velocity, vec2 texelSize) { vec3 current = texture(currentColor, uv).rgb; vec2 previousUv = uv - velocity; bool outside = any(lessThan(previousUv, vec2(0))) || any(greaterThan(previousUv, vec2(1))); vec3 history = outside ? current : texture(historyColor, previousUv).rgb; // A 3×3 neighbourhood clamp limits bright/dark ghosts when history // lands beside a disocclusion or a high-contrast edge. vec3 lo = current; vec3 hi = current; for (int y = -1; y <= 1; ++y) for (int x = -1; x <= 1; ++x) { vec3 n = texture(currentColor, uv + vec2(x, y) * texelSize).rgb; lo = min(lo, n); hi = max(hi, n); } history = clamp(history, lo, hi); float feedback = historyValid(previousUv) ? 0.9 : 0.0; return mix(current, history, feedback); } ``` Real implementations often clamp in YCoCg or another perceptual space, use a velocity-dependent feedback factor, and reject history when depth/normal/material disagree. The neighbourhood clamp is a guardrail, not a complete disocclusion solution. ### Jitter and motion vectors Use a low-discrepancy sequence such as a Halton sequence for the projection jitter, and apply the same jitter consistently to the camera projection used for rasterization and motion-vector generation. The unjittered camera is still useful for gameplay and picking; do not feed a jittered projection into [[Ray Picking Through the Rendering Pipeline|mouse picking]] unless the interaction explicitly wants the rendered sample position. Motion vectors can come from current and previous object transforms plus camera matrices. Skinned meshes need previous skinning transforms, not only previous object transforms. For a velocity at a pixel, reconstruct the current world position from depth, transform it through the previous jittered view-projection, and subtract previous UV from current UV. TAA belongs after opaque shading and before UI. Transparent particles need their own velocity/history policy or a forward composite after TAA. Bloom and tone mapping can be placed around TAA according to the renderer's exposure model; the important invariant is that the history is accumulated in a stable, linear representation and not accidentally reused after a resize or exposure cut. ## Picking a default | Project shape | Sensible starting point | |---|---| | small forward renderer, no history system | MSAA or FXAA | | forward renderer with clean silhouettes | MSAA, optional SMAA for shader/texture aliasing | | deferred many-light renderer | TAA, optionally with a modest MSAA mode for special passes | | CAD / editor view where latency and clarity matter | MSAA or SMAA; avoid history ghosting | | moving foliage / subpixel geometry | TAA with good velocity and reactive masks | | low-end / bandwidth constrained | FXAA, then measure softness before adding sharpening | Do not compare only still screenshots. Rotate the camera, move a thin fence, scroll a high-frequency texture, resize the window, and cut exposure. The failure modes appear in motion. ## Things that tripped me up - **TAA history is not free anti-aliasing.** Missing motion vectors turn history into ghost trails. - **Jitter must match the matrices.** Applying jitter to the colour pass but not to velocity, depth reconstruction, or shadow decisions creates shimmer. - **A velocity is not a depth.** Reproject first, then sample history; do not treat a screen-space vector as a world offset. - **Reset history on discontinuities.** Camera cuts, resize, teleport, FOV changes, and large exposure changes need a clear or carefully rescaled history. - **Alpha-tested geometry needs special care.** A leaf mask can change coverage without moving its vertex, so reactive masks or conservative history rejection help. - **UI should not enter scene history.** Composite it after TAA unless the UI deliberately wants temporal accumulation. - **Deferred MSAA multiplies G-buffer bandwidth.** Per-sample normals/materials can cost more than the jaggies they solve. - **Post-AA cannot fix bad texture sampling.** Generate mipmaps, use an appropriate filter, and fix shimmering inputs at their source. - **Resolve in linear space.** Averaging sRGB-encoded values produces the wrong edge colour. ## References - [OpenGL 4.6 Core Specification — multisample rasterization and framebuffer operations (Khronos)](https://registry.khronos.org/OpenGL/specs/gl/glspec46.core.pdf) - [Vulkan API Specification — multisampling and resolve attachments (Khronos)](https://registry.khronos.org/vulkan/specs/latest/html/) - [Lottes, “FXAA White Paper” (NVIDIA)](https://developer.download.nvidia.com/assets/gamedev/files/sdk/11/FXAA_WhitePaper.pdf) - [Jimenez et al., “SMAA: Enhanced Subpixel Morphological Antialiasing” (project and paper)](https://www.iryoku.com/smaa/) - [Karis, “High Quality Temporal Supersampling” (Advances in Real-Time Rendering)](https://advances.realtimerendering.com/s2014/) - [Akeley, “Reality Engine Graphics” — multisample concepts (Marschner and Patera archive)](https://www.hpl.hp.com/research/vr/book/) --- Back to [[Notes/Index|Notes]] · see also [[Deferred vs forward rendering]] · [[OpenGL - learning log]] · [[Vulkan - learning log]] · [[Compute shaders]] · [[Ray Picking Through the Rendering Pipeline]]