Modules/Sculpt-Paint-Texture/Future Projects

= Sculpt, Paint & Texture: Future Projects = (This page is still heavily under construction)

Here we list inactive or future projects & opportunities for the module before they become active projects on the Workboard. Additional docs can be linked for expanded info.

Brush Engine Refactor
This is one of the biggest blocking projects for any new feature and code maintainance. We should at some point soon focus on incrementally refactoring the entire brush and sculpting code.

Brush properties and various improvements are planned, especially for the goal to cherry-pick settings for brush assets. Could be done separately to the underlying refactor, but not wise to do so.

Sculpt-dev was used for prototyping and working out the design for the intended brush properties and unified settings. Since these mainly rely on RNA this would be doable to implement without causing to many breaking changes. But we still need to review the current implementation and design.

Moving forward we need to define a high level design first and then break them down into smaller goals.

High level design = Divide current code into logical blocks and how they need to be used. A global architecture that each tool/brush can access. How we handle the stroke, tool and resulting data. Brecht, Jeroen, Bastien and Joe have insights on this and will start to document their knowledge over 2023. Afterwards this can get a workshop and scheduled priority.

Port all painting improvements to Image Editor
With the current Paint Mode project and recent sculpt mode additions, it will become increasingly necessary to provide the same feature set to the image editor. This also relates to D5697

VDM Multires
Targets for this project are still being collected. This is a full refactor of the Multires system and how the displacement data is stored. The goal is to:
 * Make the code easier to maintain and support features
 * Fix stability issues
 * Make the displacement data more generic and easier to manipulate & access
 * Provide support for all existing features, including geometry nodes modifiers (T104262) and node group operators (T101778)
 * Fix multires baking issues

More tasks on current design topics are T84864, T93165, T88381, T81989, T80348 & T73047

More features could also be provided:
 * Refit Base (D7644)
 * Delete Lower (D7582)

Sculpting Layers & Shape Keys
The goal is to implement a layer system for sculpting subdivision levels.

A temp doc assembled with Daniel Bystedt to collect some initial ideas:

Another current limitation that could be resolved is for transfering shape keys. Currently it reads the entire topology, which makes the oepration fail if the vertex count is off. If it only reads vertex IDs, then partial meshes could be joined as shape keys. As an example: Duplicating an arm, making changes on the arm and transferring the changes back onto the entire body.

Using shape keys on alembic caches is also an important use case. Supporting shapekeys as some sort of modifier or geo nodes could be beneficial.

Specular & SSS Matcaps
Creating a Call for Content for a new set of Matcaps. This time with multi-layer EXR files and .blend files to recreate/tweak them if needed. The main goal is to add official specular (or even SSS) layer support to all matcaps and make the selection more streamlined for various workflows.

Minimal Keymap Overhaul
A small project to overhaul the default and the IC keymap would have various benefits: - Solve common shortcut conflicts between LCS and RCS - Remove all brush shortcuts for free customization - Streamlining of common shortcut operators (Multires Levels and menu operators) - Gaining consistency between different painting modes - Supporting inverted and blur strokes for Weight Painting via holding `Ctrl` and `Shift` - The opportunity to document a stronger design for the painting and sculpt mode keymap interaction

Resolution Independent Sculpting
We discussed resolution independent smoothing (D9385). This is potentially relying on the upcoming dyntopo refactor. This could be a great thing to add for easier “resolution independent sculpting”.

A resolution picker could also replace the need of the Multires modifier in certain workflows by defining a resolution target instead of switching levels.

Notes from a meeting: The smooth code in D9385 reads the same vertex coordinates it's writing to. This makes it slightly non-deterministic. We can fix by having it write to a separate buffer. This would actually speed things up quite a bit, though it'd be too much work at the moment.

Related to think about: - This is somewhat in conflict with a remesh detail size. How would the user have intuitive control over both remesh detail and sculpting detail?

Mesh target & shading options
Partly based on D8925 The goal is to intuitively be able to deform modifiers with a preview, similar to edit mode.

Especially with deformation & subdiv modifiers this can give the additional option to either sculpt on the undeformed base or a close approximation of the deformed mesh.

This could be combined with the feature of choosing between optimized viewport shading or acurate shading. (Related to T103889 & T90598)

For an optimised performance there should also be testing or options for live mesh updates or only on confirming a stroke. (Related to T103871 and T101150)

Just like with Edit Mode it would be very beneficial to be able to sculpt on a live mirror modifier results and have the stroke be correctly applied on the original base geoemtry.

Other related tasks:
 * T95761
 * T101150
 * T100585
 * T87034
 * T92372

This might also relate to how Eevee is being used and optimized for sculpt mode (D9522).

Snapshots
Based on D6594 prototype. The goal is to store a mesh temporarily as a sculpting layer or shrinkwrap target for a Morph Brush. This mesh could be saved in a clipboard independent of the undo stack.

Geometry Nodes Brushes
Partly based on D8377 and D9380. Geometry nodes could be used to simulate unique tip shapes and insert various meshes.

Plane Trim Brush
Based on D9571

Auto Retopology
The goal is to improve the automatic results of the quad remesher to immediately convert a sculpt into a multires modifier

Sculpt Mode Performance
Improving the general performance, based on T81180 and D8983.

Performance Warnings & Scheduler
Preventing freezes and crashes from certain tools and operations by either warning the user in the UI based on critical mesh statuses, or giving the user the ability to cancel the operation on long waiting times.

Texture Filters
Based on D6156 We should investigate methods to quickly and roughly insert texturing details via Filters.

Voxel Remesher Improvements

 * Better re-projection of Colors & surface details
 * Performance improvements, espeically for Fix Poles
 * Remeshing individual mesh islands without merging them
 * Remeshing selective mesh islands
 * Checking for holes in the mesh and filling them (D6111)

A wider idea is to combine the concepts of voxel remeshign and dyntopo into simply Adaptive Sculpting. This way we can re-brand the remeshers into "Volume" and "Surface" remeshers with the same shortcuts and detail settings.

The overlay could also be improved by always showing the size (D8069) or even a way to show the wires on the mesh itself instead of via the bounding box (This often causes issues where the overlay is behind the camera).

Another way could be to show the wire overlay only within the brush cursor, similar to the dyntopo density overlay and the Density brush for curve sculpt mode.

Trim Tools Improvements
The current Trim and Project tools are not performant or useful enough. But doing proper changes would need heavy reworking of the tools.

A better way of approaching the trim tools is to integrate them with the voxel remesher. This would need to be less precise than exact boolean operations and could vastly increase performance. Implement a new remesher option for extreme performance boost. Use the bool operator on a lower remeshed version and then project the higher one on that.

Project Tools Improvements
Project tools could fair geometry instead of just snapping them. Assigning them a face set like the trim tools is also a good addition.

Wrap up Gesture Tools
Add missing Line, Lasso and Polyline versions. Investigate a way to combine them into one tool. Also add them as operators to the menus.

Filter UX
Filters provide very poor visual feedback currently and controlling the strength is difficult. Gizmos based on the generic pose slider could be used here.

Modal IP mask filters from the sculpt could also be implemented if they are performant enough on high res meshes.

Transform Tools Improvements
We’ll make a task for setting the sculpt-session pivot location via a shortcut and a tool settings. Also we need to make it possible to change the orientation of the sculpt session pivot. This is closely tied to D9540

The general idea is to use the shortcut `Shift RC` to set the pivot, just like setting the 3D cursor in other modes. The behavior for setting, snapping and transforming the pivot should be the same. When using the transform operations, the sculpt-session pivot is transformed as well.

A toggle will also be added to the transform tools tool settings to only transform the pivot.

We could also consider adding a popover/dropdown in the header to change the behavior of the sculpt-session pivot: - Automatic Pivot (Expand operations change the position and rotation of the pivot, in addition to manual placement) - Manual Pivot (Only the user is able to place the pivot) - Global Pivot (An optional feature to keep the pivot location & rotation consistent across multiple objects. This could also be implemented by using and transforming the 3D Cursor together with the pivot)

Automatic Sample Bias
The sample bias for brush textures needs to be manually set. This can be easily automated by checking the value of the image borders and matching the sample bias.

More Dyntopo Support
Add more support to operators. Also dyntopo remeshing support for brushes like Grab, Pose, Smooth and Draw Sharp. Revamp or remove Simplify Brush.

Solve Keymap Conflicts
Based on T102431

Kelvinlet Tools
Fix Elastic implementation to properly preserve volume. Based on Kelvinlets paper

Improved Xray

 * Option to exclude active object from Xray transparency
 * Option to highlight Fresnel for easier reading of shapes
 * Support for mask & face set overlay

Operators only affect visible geometry
Apply this logic to Initialize Face Set operators and Grow/Shrink Face Sets.

Auto-Masking polishing
Use geodesic falloff instead of topology for boundary propagation steps. This would avoid artifacts.

We could also investigate settings to propagate only inwards or outwards of the auto-masked face set.

Incremental PBVH leaf nodes updates for brushes and operators
This could be a focus real soon as it can improve performance in various areas and fix bugs. For this we need to improve how we handle PBVH tree leaves. They need to be subdivided more. Then the actual geodesic distances could be used for checking falloffs along surfaces and making sculpting tools more aware of the mesh volume. In theory it’s possible, but needs investigation. Joe is interested.

This would have various benefits. It would help avoid sculpting on disconnected geometry within the brush radius on any brush (Not just Grab). Kelvinlets elastic brushes could be implemented more accurately by properly taking the volume of the mesh into account. Geodesic expand performance would also be heavily improved.

Menu Operator support
Based on T102427

In general operators in Sculpt Mode:

Joe worked on proper support of "Adjust Last Operation" settings. Filters, Gestures and other Active Tools were implemented in toolbar because of previous limitations.

We agreed that we want to focus some time to adding Filters, Gestures and Edit tools as menu operators, now that the previous limitations are lifted.

This would have various advantages: - Support existing [Operate -> Settings](https://wiki.blender.org/wiki/Human_Interface_Guidelines/Paradigms) paradigm - Easier to use various tools as one time operations instead of making them active - Easier to use filters via shortcuts and quick favorites. - Redo panel interactions for very high res meshes (Better with slow performance) - Simpler than implementing T97959 Color Filter: Sliding UI. That can come after.

Everyone seems to agree. Joe thinks that can be done, but mentioned that some filters already have a good performance optimization. So that is less of a reason to add them as menu operators. There might be some extra work attached to this, so what works and what doesn't needs to be tested.

Color Expand/Gradients
We should add the current operator to the menus to make it more accessible and easier to assign a shortcut. We will need to soon add a “Paint” menu in sculpt mode too, especially once more operators are added.

Further research and prototyping needs to go into a Gradient Tool for expanding gradients across surfaces, space and the screen.

Color Palette Overhaul
Various improvements were suggested for UI and how the data block works:
 * Each color slot is a saved color ID with RGB values, that can be accessed through nodes. This way complex node setups or pixel coordinates of textures can be avoided as a color palette substitute for materials & procedural systems. Instead the color palette data block is usable.
 * Drag and drop swaps colors instead of replacing them
 * Attach the color palette directly to the color picker UI (For popups)

Store Hue of Colors
Currently colors are stored as RGB values. This makes it impossible to change the Hue of a color that has 0 Saturation. (A case of sweeping away your footsteps as you move forward) We could store the Hue of a color, even if it's just in the current session.

Overlay Visibility
In any painting modes and sculpt mode some overlays are automatically hidden. While this was useful since the Origin point and the 3D Cursor or rarely if ever used, some users find them helpful. Adding support for these overlays could be implemented.

Questions:
 * Do we need to add the Cursor Tool to these modes then?
 * Should there still be an option to automatically hide them for certain modes?
 * Can this be customizable per mode?
 * Should this instead be tied to custom shading modes or custom workspaces?

Dynamic Overlays
Useful for painting to avoid clashing colors and values for mask and face set overlays. Needs more info.

Brush Behavior Overlays
Some brush settings can be very hard to understand, especially how the settings affect the brush. Additional overlays could be enabled to debug the brushes for authoring and editing their settings. Based on D6324

Geodesic falloff for Grab brush
This was already prototyped in the sculpt-dev branch. Could be implemented as a brush option. Based on this implementation D10068

Pressure sensitivity support for Anchored brushes
Based on D6603