From BlenderWiki

Layer Manager: UI Design

State: 19 August 2016

In this doc we will mainly focus on concepting a design for object layers and how to make them work with the viewport. If this design works out, we can build up on it and look into other layer types (render layers, animation layers, etc).

Note: Content of this doc is completely preliminary.

Goals

Main Goal

We want a flexible layer system for managing display of scene content. This is more than just organization, it’s about precisely defining how content is drawn or rendered.

Requirements

The new layer manager design has to meet certain requirements:

• Remove limit of 20 layers.
• Support grouped/nested layers.
• Allow adding, deleting, renaming, reordering and hiding layers/layer-groups.
• Support layer compositing (more on layer compositing).
• Allow render engines (includes viewport) to define and use layer data.
• Be ready for overrides (more on overrides).
• Provide quick access from external editors.
• Support layer(-group) filtering.
• Consistency with other parts of the UI.
• Compatibility with old visibility bit ‘layers’.
• Be artist friendly!

Additional features (nice to have, but not required):

• Layer colors from theme palette (mainly used for coloring object outlines/wireframes).
• Indicate which layer contains the active and selected objects.
• Operator to select layer elements by layer/layer-group.
• Operator to merge layers/layer-groups.

Functionality

In this section some not so obvious functionality thoughts are described. The proposal section shows concrete ideas on how to integrate them into the UI.

Layer Compositing

Layer compositing is going to be one of the main features of the layer manager. “Compositing” is a vague term, we use it to describe everything that influences the look of layer content. This can be viewport compositing effects (SSAO, Depth of Field, ...), drawing options (X-Ray, wireframes, ...), blending modes (add, multiply, ...), filters, etc.

Compositing layers are not exclusive to object layers, they can be applied to other layer types as well. E.g. if we get texture layers in future, compositing layers can be used for blending modes or filters.

Overrides

Overrides allow you to ‘override’ the settings of your content. An example use-case is to temporarily change the material of all objects in the scene.

Blender already has a small number of hard-coded overrides working on the entire scene (material overrides, lamp overrides, scene simplifying), but a more generic solution is needed, which allows you to precisely specify values or entire data blocks to be overridden The nature of layers - having grouped content - allows them to work as base for an overriding system, whereby you can apply overrides to entire layers, layer groups or only to a part of the layer content (selected using filters).

Note: Some artists raised the point that it would be nice if overrides could change the object entirely to allow some sort of versioning. For example the layout department could apply an override that replaces an object with a low poly version. However, this might fit better into the asset management, but it’s worth considering. Feedback welcome.

Layers and Render Engines

Another key point of the new layer manager will be its role in the rendering pipeline. Layers will be a bridge between content and render engines. Thus, layers have to communicate with the render engine: The render engine will register properties which it passes to the layer manager for display (more on layer properties later). The layer manager can then provide the layer content and the current property values, which the render engine can eventually use to render the final image. Further, render engines need the ability to define own compositing layers and pass them to the layer manager.

Viewport as Render Engine

It’s important to note that the viewport is going to be handled as a regular render engine. Especially as the new viewport will allow workflow based drawing methods such as game engine style PBR, optimized drawing for fast animation playback, mesh drawing for architectural visualization, etc. Layer properties (again, more on layer properties later) and compositing layers can be used for defining and adjusting these drawing methods then.

Layer Data Interpretation

How exactly the data is interpreted or used to create the final image is up to the render engine. The layers just provide a graphical user interface for the properties and internally some utilities for getting information about the layer content. One of the advantages of a proper layer stack is that layers can be rendered one by one. Render engines could render individual layers as passes and “alpha over” them for the final image. Such passes could later be re-composited but they can also be used to avoid re-rendering of layers if content hasn’t changed.

Layer Properties

To hook up a render engine with the layer system, it has to tell the layer manager which properties to show for a layer type. Such properties from the render engine can be: Display type (wireframe, solid, bounding box, ...), compositing effects (ambient occlusion, depth of field, motion blur, ...), etc.

Layers themselves provide a bit of (predefined) scene level data. For example:

• Content of the layer (objects)
• Theme setting based layer color which the render engine can use for display (e.g. for colored wireframes)
• Restriction flags (visible, selectable)
• Dependency graph flags (e.g. skip dependency evaluation)

Current Render Layers

Bringing layers more closely to the render engine might also make our current concept of render layers redundant. Render layers could be integrated with the normal object layers. However, we don’t want to do this, just for the sake of having an all-in-one layer management solution. It has to bring actual benefits for artists. We’re still a bit unsure what the best way to go is, so we decided to go on with the work on regular object layers first and then check if merging them with render layers makes sense.

Scene vs. Local Level

Layers are going to be scene level data. This means if you change a property of a layer, it’s going to have an influence on every editor of the scene. However, since users may want local variations, for example different drawing per viewport, there needs to be a way to make local exceptions, local overrides actually. The proposal section contains an idea of how this could work.

Workflow Integration

Ideally the new layer manager would integrate nicely with the idea of workflow modes that are planned for the 2.8 project. Unfortunately it’s still not clear how those will look exactly, so we can’t really make any decisions yet. One idea to keep in mind for this is allowing the workflow modes to add compositing layers as a preset.

Grease Pencil Layers

Layers are used in the current grease pencil workflow as a container for strokes. Instead of having to change attributes of the individual strokes, the user can simply change attributes of the layer, which then affects all strokes inside. This current workflow for grease pencil strokes in the 3D view doesn’t really fit into the regular workflow for 3D content though. It’s similar in some regards, but quite different in others.

Now, to fix this, the grease pencil has to be integrated more closely to objects. The idea came up to add a grease pencil object type, which would contain the individual strokes. It would be a proper replacement for the current grease pencil layers, but playing nicely together with common workflows in Blender. The user could put the grease pencil objects into the regular layer stack.

Objects on Multiple Layers

The current, 'old' layer system of Blender allows having objects on multiple layers. The question has to be asked if this functionality should be kept. It was useful for things like restricting light or collision influence to objects on the same layers. Groups, or more precise selection methods could be used for this however.
Many users also used it to manage multiple object setups that share some objects. This seems like a workaround for missing render layer support in viewport, so a proper solution would be adding this support.

So as you can see there's no real need to keep supporting objects on multiple layers. We'll try to remove it in favor of the proper solutions mentioned above.

UI Proposal

Time to take the requirements and the previously listed thoughts on functionality to create an actual UI design proposal.

Location

Main (Scene) Layer Manager

The outliner is made for scene organization and inspection, so having the layer management there too seems to fit well. It’s still an option to use an own “Layer Manager” editor for this, but currently there doesn’t seem to be a need for it.

Local Layer Setting Overrides

As said earlier, users often want local variations of layer settings. For example different compositing settings to produce different looks in different viewports. For this, more lightweight lists of the layer manager can be displayed, which allow locally overrides for settings of the scene layer manager. They can be moved into regular panels. Changing a setting there would override its global (scene-level) value. To visualize that a value is overridden, it can be displayed in a different color. The context menu of the button would give an option to use the scene value, which simply deletes the override.

Just like said earlier, overrides need to be designed as a separate project. What’s described here is a rather primitive idea, just to allow us making local overrides.

As an alternative to local overrides, we can also make the local layer lists mostly independent from the main layer manager, but provide operators to sync or copy values from it.

Keeping Old Visibility Bits

For some people this may be good news: We propose to keep the old visibility ‘layer’ bits, the ones represented by the 20 little buttons in current Blender:

Note: They used to be called “layers” which they aren’t really. We should stop thinking about them as layers and call them differently. In this doc we call them "visibility bits".

Instead of being a scene level property, they’re going to be a layer property, possibly with the option to use any number between 1 and 64. So within each layer, you can assign each object to 1 to 64 visibility bits.

The visibility bits of the active layer can be shown in the 3D View header (or anywhere else obviously), solving the need of a way to quickly toggle layer content visibility.

Technical diagram on how this would work here (by Ton Roosendaal).

Layer Settings

The layer settings are the place that lists the layer properties, either coming from the layer itself or from the render engine. They are going to be in a collapsible part of each layer. To open/collapse it, a gearwheel icon can be displayed on layer mouse hover and/or on selected items (to avoid visual clutter).

Layer Compositing

While it’s okay to make some of the layer compositing features simple layer properties, users may often want to apply them to multiple, or even all layers. Instead of making them all layer properties, a special layer type can be introduced, “compositing layers”. A compositing layer is applied to all layers that are lower in the hierarchy (hierarchy described later), to make them affect a single layer only it should be possible to attach them to a layer. It would then be a nested compositing layer. This will also allow local visibility overrides, meaning the user can disable compositing layers in individual editors.

The question if a compositing effect should be a layer property or an own compositing layer is a per-case decision, if users usually want to apply it to multiple layers and/or if it has a number of options to display in the layer settings, it’s better to give it an own layer type. Otherwise a simple layer property should be preferred.

Overrides

Just like we propose to add special compositing layers, it's an option to add 'override layers'. Possibly this is over-complicating things though, so we might be better off going with a more primitive solution.

We decided to postpone detailed override design discussions for the time being. They should be handled as a separate project.

Note: Some artists raised the point that it would be nice if overrides could change the object entirely to allow some sort of versioning. For example the layout department could apply an override that replaces an object with a low poly version. However, this might fit better into the asset management, but it’s worth considering. Feedback welcome.

Conflicts

Hierarchy

To avoid conflicts, layers should follow a clear hierarchy. Such a hierarchy will define how conflicting settings of different layers/layer-groups are to be handled.

In Blender it’s convention to build orders from top-to-bottom, whereby the uppermost element is the first element. For the layer manager, this might be a bit unintuitive at first, since people are used to layer managers that work the other way around. Even though this being unintuitive is an issue we recognize, we don’t want to break consistency either. Switching Blender’s top-to-bottom convention to the bottom-to-top approach is an option to look at during the 2.8 phase. For the time being, we’ll stick with existing conventions and use the top-to-bottom order.

If we allow layer grouping, grouped items have a lower hierarchy level than the group they belong to. From this it follows that the group settings override the settings of its items.

Mode Conflicts

The object modes (object mode, edit mode, pose mode, ...) cause conflicts with the layer system. For example, if you hide the layer containing the object that’s currently in edit mode, you’re left in edit mode without a visible active object. The current visibility bits are just not drawn in edit mode to avoid such conflicts, so we need to find a proper solution.

A simple solution would be to just switch to object mode and deactivate the active object if needed, whenever there’s a mode conflict. Alternatively, each layer can store an active object and a mode for it. In example above, the previously active layer will be activated again and the active object of it will be switched into the mode it has been in earlier (object mode by default). It will also not be possible to move individual polygons to different layers.

Discussing this, there was quite some consensus that the simpler solution of always switching to Object Mode on conflicts is preferred.

Open Questions

They’ve mostly been touched above, but there are a few questions to be answered still:

• Is support for objects on multiple layers still useful? (Probably most urgent question)
• Where can I buy it?! :)

Questions for Later

• How can workflow modes make use of the layer system?
• Should we integrate this new layer manager with light linking? If yes - how?
Ideas:
• Add layer type that restricts the illumination of objects/lights to layers lower in the hierarchy or the layer it’s attached to.
• Use object level option from Blender internal to illuminate only the layers the object is in. (For this option we should really keep the abilitity to have objects on multiple layers)
• Show layer list in lamp settings to select which layers to include/exclude.

Note, in Cycles there is basically no difference between lamps and objects, since objects emit light too, due to indirect lighting and support for emission shaders.

• How exactly would a UI for overrides work and look?

Icons used in mockups: Font Awesome by Dave Gandy - http://fontawesome.io (License: SIL OFL 1.1), Subway Icon Set by Mariusz Ostrowski - (License: CC BY 4.0)