From BlenderWiki

Branch Overview: particles-2010

This is an overview of the bigger changes and sub-projects implemented in the particles-2010 branch. For the purpose of merging with trunk, all the particle and modifier trees and nodes have been stripped. There are still many small changes and additions that don't directly relate to any of these projects, but here you will only find the bigger ones.

Internal Changes

Type Structures and Templates

• Node Trees get their own type struct

Node trees were defining specific behavior by switch statements in core node code, which makes code hard to read and maintain, especially when adding new tree types. Now trees get their own type struct (bNodeTreeType) as well, which makes core code more generic.

• Node socket type meaning changes

The existing bNodeSocketType struct was used to define static lists of sockets used for a node type. This struct has been renamed to bNodeSocketTemplate now. A new struct named bNodeSocketType has been added, which works much like node and tree types, providing basic parameters (such as the circle color) used for a socket of that type.

• Node templates

bNodeTemplate is a simple collection of data that's needed to define an actual node. For group nodes it also holds a pointer to the node tree that makes up the actual group type. Can be extended by other parameters in future if required. Avoids a long list of unused arbitrary parameters in the API for adding nodes.

Socket Input Buttons

Old sockets had a bNodeStack struct to store input button values. This is too restrictive when adding new socket data types, e.g. ints or matrices. Sockets now have a generic void *default_value, which points to one of several simple structs bNodeSocketValueInt, bNodeSocketValueFloat, etc. In addition to the actual input value, these also store custom limits and a RNA property subtype (if it's meaningful for the data type). This way node buttons can make full use of the new interface possibilities.

Display vs. Execution order

Nodes were previously always sorted in their execution order, even in the UI. The only exception was that selected nodes were drawn on top of unselected nodes. As part of a new "frame" node feature (#Frames), a stable Z-order of nodes has been implemented. Front/back ordering does not depend on node links in any way now, only selecting nodes will bring them on top of others. During execution a separate list of nodes is used, which is sorted by dependencies as before.

File structure changes

A couple of new files have been added to better organize the code.

• Each tree has its own file now (e.g. CMP_nodetree.c), to define the tree type struct.
• Group node code is now in its own file (currently ***_common.h/.c, in order to share static functions with loop nodes). The idea is that the core code should not contain any reference to specific node types or tree types. All special behavior of group nodes should be behind the abstract interface provided by the node type struct. (Note: this is still ongoing work, some parts may need to be cleaned up further).
• node_socket.h/.c contains socket type definitions.
• Shared functions for execution of cmp/mat/tex nodes are in node_exec.h/.c. These are basic functions that define socket stack indices and prepare threaded data stacks. In the future the current tree types will be replaced by more sophisticated implementations, at which point this file will be obsolete.

The file structure could be improved further by creating own directories for each of the tree types. I did not do this yet simply because it breaks each and every merge operation with SVN when one of the nodes is changed.

Execution Data

One of the most problematic design parts of the old code is the close integration of execution data into the node trees: Each tree directly stores all data produced during execution (in the stack and threadstack pointers). This is a big problem because:

• It won't work for advanced tree types, which have no use for the old, simple bNodeStack data storage.
• It is bound to break parallel execution. While it still works for threaded sample calculations in material nodes, it causes trouble with node groups and possible multiple calls for the same tree.

The solution is to keep execution data strictly separate from the node trees themselves. Most new node-driven features already apply this principle by adding an intermediate node representation, which is "translated" from the original tree in a preparation step. This resembles a compiler translating a source code file (the visible node tree) into machine instructions (the internal execution structures).

At this moment the node tree still stores a pointer to the classic execution data used for compo/material/texture trees (collected in a single bNodeTreeExec pointer), for the sole reason that the render pipeline currently does not store execution instances, but needs to get those from the material or texture (via the node tree).

Groups inside Groups

With the new execution instance approach group nodes have gained the very important ability to create groups inside other groups. A generic callback in node types can be used to check a node template for validity before an actual node is created. This is implemented for groups to avoid recursive group hierarchies. Real recursion can still be implemented using a specialized group type with a fixed recursion limit later on.

Instead of "inlining" groups by including the execution graph of the group into that of it's parent tree, the group now stores its own execution graph (i.e. bNodeStack data) for the internal tree. Since there is one such execution instance for each group instance, multiple groups can run in parallel without interfering.

The UI currently has no good way of editing groups inside other groups. The only way to access an internal group is to make a second instance of that group on the top level (outside the parent group) and then edit that top-level instance.

The parent group

Workaround for editing the internal group

Hidden/Disabled Features

Loop Nodes

These are an experimental feature, which is currently disabled. Loop nodes allow you to repeat parts of a node tree by means of a specialized group-style node. Inputs and outputs of loop trees are mirrored, so that the output result of one iteration can be used in the next iteration.

Unfortunately the old tree execution system is not designed for this kind of repetition and is not working nicely with loops. The new node tree projects should take this kind of flexibility into account right from the start.

Quick Viewer Output Buttons

Each output socket has a flag with an associated button for viewer output. This replaces the current method of adding a special viewer node, that has to be linked to the output in question. The new button is a much simpler and faster way of defining the viewer output. The final output used in the render/material/etc. should still be defined by an output node, as this is more flexible and doesn't have to change so often. Current tree types do not use the output flag yet, so the button is disabled to avoid confusion.

Frames

Frame nodes are a special new node type without any actual execution function. They are a pure UI layout helper feature, which is meant to provide lightweight grouping features. Nodes that are dragged on top of a frame node are then attached to that frame. This way frames are an easy way of grouping nodes without creating a full blown group node type. Frames always stay behind other node types (but keep their relative order like all nodes do now). They can also be attached to other frames, effectively building a hierarchy of frame nodes.

Here's a video demonstrating frames:

Socket Panels

This is a small experimental feature to bundle sockets for large nodes. Above each socket panel a label with a collapse/expand button is displayed. This feature would only be useful for extremely large nodes with a lot of sockets. It is not in use currently, but does not do any harm (UI-only feature). Could probably be removed.