User:Sergey/DraftMultiresolutionSurface

Draft: Multiresolution Surface

Draft This is a draft. Once the text and images are finalized the page will be moved to the official code documentation section.

This document gives a quick dive into what multiresolution in Blender is, gives a quick overview of top-level algorithms, and gives an idea where to look for the parts of those algorithms in code.

Base mesh - mesh which is an input to the multiresolution modifier. It might include other deformation modifiers (if there are any leading the multiresolution modifier)..

Grid: a special representation of high-res subdivided mesh in Blender. This is done by only storing coordinates of high-res vertices, without the full specification of edges and polygons between them.

Each coarse mesh polygon consists of as many grids as the number of loops in the polygon. All grids are met at the median of the polygon. The grids are placed as:

• One vertex is on the polygon median.
• Two vertices are on adjacent edge’s medians.
• One grid corner matches the polygon’s vertex.

Grid topology for regular (quad) and non-regular (triangular) faces (in blue)

The coordinate space of a grid starts at the center of the polygon.

• U axis of the grid points towards an edge connecting the current loop with the next loop.
• V axis of the grid points towards an edge connecting the previous loop with the current one.

Grid coordinate system

PTex face: a quad face which comes from polygons of a coarse mesh. Quad coarse polygon consists of a single ptex face, Non-quad coarse polygon consists of n ptex faces, one for each loop of the polygon, the ptex indices within the polygon are aligned with loop indices. For the non-quad polygons, the ptex face vertices are placed as following:

• One vertex is on the polygon median.
• Two vertices are on adjacent edge’s medians.
• One ptex face vertex matches the polygon’s vertex.

PTex face topology for a coarse mesh (in blue)

Coordinate space within ptex face goes as following:

• U axis is pointing from the coarse polygon vertex in the direction of the next vertex of the polygon.
• V axis points in the direction of the previous vertex of the polygon.

For the quad polygons the origin of the ptex face coordinate system is at the first vertex of the polygon. For non-quad polygons the ptex coordinate system’s origin is at the ptex vertex which corresponds to the coarse polygon vertex.

Coordinate system of PTex faces

Note that this is close to the idea of Blender’s Grids, but with different behavior for quad polygons and different coordinate systems within the grid/ptex face.

Limit surface: a surface which corresponds to the coarse mesh with an infinite number of subdivision steps applied.

What is multiresolution surface

Multiresolution surface is effectively a combination of a subdivision surface and a displacement map.This allows to keep adding details to the model without affecting its base mesh. It is possible to view the multiresolution surface at any level of subdivision.

Displacement grid

Multiresolution surface uses displacement maps to store displacement. These maps are aligned with subdivision grids. They are storing per-highres vertex displacement in a tangent space of the base mesh.

All the grid have the same dimension of grid_size = (1 << (subdivision_level - 1)) + 1 elements along each side.

The grids are allocated to the maximum subdivision level of the Multires modifier. When it’s needed to visualize mesh on a lower level then only a subset of information from grids is used. The following picture visualizes which displacement grid elements are used at different viewport levels for top multires subdivision level 3. Note that the displacement grid is aligned with subdivision grids, which are stored per-face-loop.

Displacement grid at different multires and viewport resolutions

As was mentioned above, displacement grids are stored in the tangent space of the base mesh. This allows to deform the base mesh without destroying details added by the multires modifier. This is used, for example, on animated characters.

The tangent space is calculated based on OpenSubdiv’s evaluator, which gives position on the limit surface and first derivatives in PTex’s face U and V direction. Let's call those P, dPdu, dPdv respectively. Since the UV space of a PTex face might not match the space of which corresponds to this PTex face (grid might be “rotated” in respect to the PTex face), some conversion is needed. Additionally, the derivatives are to be normalized and put into a transformation matrix. This is all done in the BKE_multires_construct_tangent_matrix() function. Let's call the result of this function tangent_matrix. Now simple math can be used to find final vertex position:

 object_space_displacement = tangent_matrix × tangent_space_displacement final_position = P + object_space_displacement

This is done for every vertex in the final mesh and its corresponding grid element.

Limit surface and tangent space

There are two factors which are to be taken into account when calculating tangent space for multires displacement:

1. Displacement grid storage is not hierarchical: for any given vertex in the high-resolution mesh there is only one displacement defined by the grid (in other words, displacement grids define vertex displacement on all levels).
2. Changing a display level of multires should only add/remove details, without changing the way the model looks like.

This leaves the only option to calculate the tangent space: use the limit surface of the model, so that tangent space stays the same on any given point on the surface regardless of multires subdivision level. This decision will later affect the way how propagation and subdivision works, but it allows to avoid drastic model changes (and even artifacts) when changing current display subdivision level.

Displacement grid averaging

There are vertices along grid boundaries which have corresponding displacement from different grids. For example, base vertices of the default cube have 3 adjacent grids, and hence 3 displacements in the tangent space.

For historical reasons these displacements are to be averaged in object space in order to give the correct final result as it is not guaranteed that they define the same displacement in object space.

Displacement grid stitching

Some algorithms (like Smooth brush in Sculpt mode) are working on grid level, without looking into inter-grid connections.This could cause discontinuity in displacement maps: for example, without doing anything extra, smooth brush will separate grids of the same face. In order to prevent this grid stitching is to be performed.

This is done in BKE_subdiv_ccg_average_stitch_faces()

NOTE: While this isn’t directly related to the displacement grids in tangent space, this is very much related.

Reshape

Reshape is a common class of operations which affects on the way how the multiresolution surface looks like. The files are situated in blenkernel/intern/multires_reshape*.

Reshape from object

This operation makes it so that the active multiresolution object is shaped the same way as the selected one..This is done in the following steps:

1. Assign grid elements to object-space coordinates
2. Propagate changes to top level, if needed
3. Convert grid elements to be a displacement map in tangent space

Code reference: multiresModifier_reshapeFromObject()

Apply deform modifier

This operation applies deformation modifier on multires surface rather than on the base mesh, Internally it is implemented in the same way as reshaping from an object, with the difference that the source for the final vertices locations in object space is coming from the deformation modifier.

Code reference: multiresModifier_reshapeFromDeformModifier

Apply base

The goal of this operation is to cancel out displacement at multires level 1: in other words, make it so multires level 1 is the same as base mesh + subdivision level 1. The algorithm goes as following:

1. Store current multires surface in the object space
2. Modify base mesh
3. Re-shape to stored multires surface state to ensure no distortion happened

The step which desires more description here is the modification of the base mesh.This is done in the following steps:

1. Assign base mesh vertex locations to final positions of corresponding vertices of the multires surface.
2. “Re-fit” the base mesh, inverting effect of subdivision surface.

The refitting is needed because the final vertices locations are only known after the subdivision surface has been applied (as a process of multires modifier). This process is a heuristic, which attempts to give good enough results, but is never perfect. The idea of the current implementation is to push every vertex away from the center of mass of adjacent faces in the direction of the average normal of these faces.

Code reference: multiresModifier_base_apply

Propagation

The propagation is used when displacement grids have been modified on a non-top level and to make it so all changes are nicely applied all the way up to the top level. For simplicity let’s consider sculpting on sculpt level 2, which is lower than the top level 3. This is visualized in the following picture.

Displacement grid vertices at sculpt level (green) and at top level (red)

The green circles indicate changes done on the sculpt level, red ones correspond to displacement on level 3. Sculpting tools will not modify level 3, so if propagation is not done then the model will look spiky.

In general words the propagation will smoothly interpolate changes made on sculpt level and add them to all higher levels up to the top one.

For the simplicity of intermediate calculations propagation is happening in the object space. The propagation itself is based on subdivision surface created using sculpt level as coarse topology. This topology is refined and evaluated for original (non-modified sculpt level) and modified sculpt level. The delta between subdivided surface with modified sculpt coordinates and the original coordinates defines how much displacement is to be added to the original grids on all levels. The simplified equation is:

 displacement_grid += subdivided modified sculpt level - subdivided original sculpt level

Ofcourse, this is done on per-displacement grid element bases (hense subdivision to level top level - sculpt level, to make elements align in count). And finally the new displacement is converted from object space to tangent space.

Code reference: multiresModifier_reshapeFromCCG(), multires_reshape_snooth_object_grids_with_details()

Subdivision

Adding subdivision levels to multires is based on using top subdivision level and calculating subdivision surface on it. This allows to smooth sculpted details and propagate them to the new level, but it has the downside of the non-sculpted object keeping shrinking when adding subdivisions to multires.