From BlenderWiki

Jump to: navigation, search

Note: This is an archived version of the Blender Developer Wiki. The current and active wiki is available on wiki.blender.org.

Note

This is a research page, actual implementation hasn't started yet.

Tangent space normal maps is a system to store normals at arbritary points in the tangent space of an arbitrary face. The advantage of this method is that an object can be deformed, and the normals will still be correct, unlike more simpler methods such as object space or camera-space normal mapping.

Constructing the Tangent Space Matrix

The tangent-space consists of three vectors. The normal that explains what is up, and two vectors (perpendicular to the normal) that is called the tangent and the bi-tangent. They can be defined for each point on a mesh and taken together they define a coordinate frame. If they are put in a matrix the matrix is called the tangent-space-matrix. Such a matrix can constructed for each point on the surface of a mesh.

The construction of the tangent-space-matrix is easiest to explain using a two-pass approach.

Pass 1. Finding Tangent-Space for vertices

In the first pass the tangent-space-vectors for each face of the mesh is found. If the object is "smooth" the tangent-space-vectors can be averaged over all the faces that share a vertex. This is similar to how "smooth" vertex-normals are calculated from the face-normals of a mesh.

If the "smoothed" tangent-space-vectors are used a normalmap that is entirely (0,0,1) (blue if colorcoded) will yield the usual interpolated normal perfectly. Otherwise the usual normal interpolation has to be coded in the normalmap.

See here and here for info on how to form the tangent-space-matrix for a face. Some code for averaging the per-face-tangent-space-vectors can be found here.

Note that extra care has to be taken were the UV-coordinates of a mesh "jumps" and at places were mirroring occurs. In general only vertices that shares UV-coordinates are averaged over. But the original smooth normals should be used. To get it all together Gramm-Schmidt can be used (it takes three almost orthogonal vectors and make them orthogonal, keeping the first (normal) intact).

A nice tutorial on some issues creating the TS can be found from the GDC2007-talk by Martin Mittring/CRYTEK. They also have source code in their CryEngine SDK. See under Resources.

Pass 2. Finding Tangent-Space-matrix for an arbitrary point on a face

The tangent-space matrix for an arbitrary point in a face can be found by linearly interpolating the vertex-tangent-space-vectors, normalizing them and then putting them in a tangent-space-matrix. When to normalize is implementation dependant. The interpolation can be done using barycentric coordinates.

Creating Tangent-Space Normal Maps

Ray Baking

Usually creating normal maps is done by casting rays from a low-poly mesh to a high-poly one. For each ray in a triangle, the corrusponding normal of the intersection between the ray and the high-poly object is computed. This normal is then multipled with the inverse tangent space matrix to form the tangent-space vector that is stored in the normal map.

Available tools

ATI's normalmapper (3dsmax-plugin or standalone) - link
NVIDIA's normalmapper called Melody - link
Granny 3D - link

Random Noise Splatting

You can also splat different tangent-space maps across a texture, to create random noise, or dirt, or scratches, etc. This can be done in any good image manipulator.

NOTE: Since blue is always "up" it will just work if it's a tangent-space-normalmap.

Usage of normalmaps in games

From the perspective of a game a normalmap is something that is magically created in the "art pipeline", prior to executing the game. The "art pipeline" could be part of a 3rd party program or an in-house tool for game-level-construction. Often some form of batch-processing of data is performed to create assets such as normalmaps.

When rendering a triangle it's vertices are fed to a vertex-shader that calculates per-vertex-attributes. The per-vertex-attributes are then linearly interpolated over the triangle of the triangle and fed to a fragment/pixel-shader. A pixel that is to be shaded (has its color determined) is called a 'fragment'.

NOTE: Remember that extra care must be taken when transforming normals if non-uniform scaling is present in the transformation matrices.

OS - object-space
TS - tangent-space
WS - world-space

Object-space normalmaps

Out from vertex shader: UV-coordinate for normalmap.

For each fragment an OS-normal is fetched from the normalmap using the UV-coordinate. The normal is then transformed into WS and used in the lighting/material-equation.

Parts of the lighting/material-equation can be done per-vertex and some per-fragment.

Pros

The usual per-vertex normal could be entirely ignored since it's not used.

Cons

Since the normals in the normalmap are specified in OS the out-direction will differ depending on where you are on the mesh. Since normals are mostly pointing outwards the OS-normalmap will be quite colorful.

For an example of a colorful normalmap, take a look at the horse here.

Tangent-space normalmaps

Out from vertex shader: UV-coordinate for the normalmap and 3 vectors in WS (normal/tangent/bi-tangent).

We assume that the normal/tangent/bi-tangent are stored in OS and transformed into WS by the vertex-shader so that the fragment-shader get them in WS. They might have to be re-normalized in the fragment-shader to maintain their unit length.

For each fragment a TS-normal is fetched from the normalmap using the UV-coordinate. The three vectors forms the TS-matrix. The matrix describes the transformation from WS to TS. The inverse takes vectors from TS to WS. Using the inverted TS-matrix the TS-normal is transformed into WS and is then used in the lighting/material-equation.

Parts of the lighting/material-equation might be done per-vertex and some per-fragment.