From BlenderWiki

Jump to: navigation, search
Note: This is an archived version of the Blender Developer Wiki. The current and active wiki is available on wiki.blender.org.
WIP
This documentation includes changes from the customnodes branch. It will be moved to the official developer docs when the branch merge is imminent.


DNA/BKE Structure

The node system consists of 4 main components:

  • Node Trees struct → bNodeTree
  • Nodes struct → bNode
  • Node Sockets struct → bNodeSocket
  • Node Links struct → bNodeLink

bNodeTree

Node Trees are the main data block container for nodes. The DNA struct is based on the common ID struct.

Node Tree DNA structure

Node tree data blocks are used in two different ways in the rest of Blender DNA data:

  1. Integrated bNodeTree data blocks
    Integrated into another ID data block, such as Scene, Material or Texture. This is a legacy feature in compositing, shading and texture nodes.
    The parent data block creates and frees the internal bNodeTree. Integrated node trees can not be linked to a node tree from the database. This makes it easier to handle some types of nodes, such as RenderLayer which need a direct pointer to their parent data block. If such node trees were linked in multiple places they could not refer to one specific scene.
  2. Linkable, "true" bNodeTree data blocks in the main database list
    Like other data blocks, node trees can be linked to by a pointer in another data block. Currently this is used only for node groups. See Node Groups for details.
to do
 The node tree list in the database API is called "node_groups" currently. This is because the only place where node trees could be used as true, linked data blocks were the node groups. However there is no reason why node tree data blocks could not be linked in other contexts, so this may be changed to more generic "node_trees" in the future.

Nodes List

Nodes are stored in a standard linked list in the bNodeTree struct (nodes member).

The order of nodes in the nodes list defines the drawing order (Z-order) of nodes in the editor window, back to front (last node in the list is drawn on top of earlier nodes). Selected nodes are moved to the back of the list, the active node is always the last, so it is drawn on top of all others.

Earlier Node Sorting
In earlier versions of Blender (up to r35751) the node list was sorted according to topological dependency based on links.


Links

Node Links are stored in the links list (storing bNodeLink). They consist simply of two (Node, Socket) pairs to define connections between nodes, where (fromnode, fromsock) is an output and (tonode, tosock) is an input.

In addition node links store a validity flag. Invalid links are displayed in red in the node editor and are ignored during evaluation of a node tree. The exact conditions for valid links are defined by the node tree type (see #NodeTreeLinkValidity). Some typical conditions are:

  • No cyclic links: Connections from a node's output to any of its inputs are not allowed (also via intermediate nodes).
  • Mismatching types: If data types of two sockets cannot be converted into each other they can not be connected (Note: Standard node socket types in Blender currently always allow conversion)

Some operators (e.g. link dragging) can immediately undo their result if it leads to an invalid link. In general though an operator may produce ambiguous invalid links, in which case the user has to undo or clean up invalid links manually.

bNodeTreeType

Each node tree has a specific type. The type data is stored in a runtime struct bNodeTreeType, referenced by the typeinfo pointer. This pointer is initialized after loading a .blend file or appending/linking from another file.

Node types can be registered by using the API. More information on registering node types from Python can be found under #PythonNodes.

The bNodeTreeType struct also links the node tree to the associated RNA struct type, via ExtensionRNA.

Node tree type information consists mostly of a number of callback functions which define the shared features of this type and the relations of nodes in the tree. More information and examples can be found in the API documentation.

Type Identifiers
Older versions used a fixed integer type identifier ("type" member). While this is still in place for backward compatibility, the primary method of identifying the node tree type is now string-based ("idname" member).


bNode

Nodes are the primary entity to define a node tree's function. They are displayed as boxes in the node editor and can be moved around independently.

Each node has a type, which defines the data it stores and how it is displayed. See #bNodeType for details.

Node DNA structure

Data Storage

Nodes can store any number of properties, for which there are two available options:

  1. Generic void *storage pointer in the DNA
    The storage pointer requires modifying the DNA files and adding specialized read/write code, old static node types use this exclusively. It allows storing all possible kinds of data including complex structs (e.g. curves and color ramps), but requires changing the Blender DNA and is therefore inaccessible to extensions.
    In addition to the storage struct there are a few legacy data members in the bNode struct
    1. ID *id: stores a pointer to a generic ID data block, such as Material, Scene, Image or bNodeTree (for node groups). The id pointer has some automatic management for file loading and user count, so nodes don't have to implement it individually.
    2. custom1..custom4: Simple float and int properties can be stored here. Used by some nodes to avoid small extra DNA storage structs.
  2. Custom properties
    Custom properties can be defined using the API, which makes them a better option for dynamic extensions of the node system. They use the ID property system automatically for storing values of the RNA properties. Properties can be defined for any RNA type in C or Python code at runtime and will be stored in DNA data without the need to define structs in makesdna. See #PythonNodes for further information on how to define node types using the bpy API.
ID Properties Limitations
Currently the ID properties system has a few limitations:
  • Pointers to data blocks can not be stored directly. A workaround is to store library data paths as strings.
  • Some existing data structs, such as color ramps and curves, cannot be created directly as ID property groups. This will need further work.


Socket Lists

The power of nodes comes from the ability to connect them with each other using sockets and links. By connecting nodes with each other, input data can be manipulated by nodes in a specific order and nodes can use the results of another node's calculation.

Each node can define a list of inputs and a list of outputs, both of which use the bNodeSocket struct (for internal structure of bNodeSocket DNA see #NodeSockets). Each link connects exactly one output socket to one input socket. Depending on the connectivity limits of the node systems, a node socket can be connected to multiple other sockets by several links.

Connectivity
The standard connectivity type in compositing and shading nodes allows multiple links per output socket, but only one link per input socket. Future node systems may allow different kinds of connectivity


Drawing

Nodes display their properties either inside the node box directly or in a separate properties panel for the active node. The latter method should be used in cases where the properties layout would take up too much space in a single node or when properties are considered advanced and don't need frequent user interaction. The draw functions of a node are part of its type definition

bNodeType

Nodes link to a bNodeType struct for type information.

Type Identifiers
Older versions used a fixed integer type identifier ("type" member). While this is still in place for backward compatibility, the primary method of identifying the node type is now string-based ("idname" member).


Type Storage
In older Blender versions the node types were stored as lists inside a specific bNodeTreeType on registration. This was an unnecessary limitation, effectively restricting use of a node to just one type of node tree. Node types are now registered in a single global hash. Any limitation of node instances to tree types should be part of the poll function.


bNodeSocket

Node sockets are simple data structures to define an input or output socket of a node. They have a unique identifier string in addition to the UI label, so that multiple sockets with the same label can be distinguished.

Sockets are connected to each other by the bNodeLink list in node trees (see #Links).

"link" pointer
bNodeSocket has a pointer called "link", which can be used for input sockets. However, this feature is flawed because it relies on a single input link, which only works with the standard 1-to-many connectivity model and only for input sockets. The pointer is also not valid in all cases (only after updating), so best practice is to avoid it and use the node tree's link list directly. Might be removed at some point.


Properties

Socket properties can be stored using the custom properties system. Note that these properties are defined for a node socket type and are not specific to a particular node type. Nodes can still use a socket's ID properties directly for storing individual data, but a preferred method is to store data in the node and use the socket identifier to map a socket to these properties.

to do
 Example?

bNodeSocketType

Node Sockets link to a bNodeSocketType struct for type information.

Type Identifiers
Older versions used a fixed integer type identifier ("type" member). While this is still in place for backward compatibility, the primary method of identifying the node socket type is now string-based ("idname" member).


The socket type defines a few details about how a socket is displayed in the UI, including the color of the connection symbol (socket circle) and draw functions for the default input value (see #InputValues).

Interface Types

A secondary RNA type definition for group nodes. These types define the necessary properties to generate interface sockets in a group node, such as the socket name, type and type-dependent details (e.g. input value subtype, range, default, etc.). Instances of these types are not used in regular nodes, but only in node group trees as part of the interface.

Standard Socket Types

Blender defines a couple of socket types used by the shader, compositor and texture nodes. These socket types could be used in new node systems as well, but they can define customized socket types if needed.

The standard socket types defined in bpy are:

Socket Type Reference
NodeSocketFloat Floating pointer number
NodeSocketInt Integer number
NodeSocketBoolean Boolean (true/false) value
NodeSocketColor RGBA color
NodeSocketVector 3D vector
NodeSocketString Text string
NodeSocketShader Cycles shader (closure)

A common feature of data socket types (numerical and string data) is to display a default input value for unconnected inputs. The node evaluation system then uses these constant values instead of results calculated by another node. These input values can be stored in two ways:

Different Designs
The designs for input values described here are in no way hardcoded or a required feature. They are just used for the existing standard socket types for numerical and string data. New node systems can define a completely different set of socket types and implement input values in a different way if needed.


  1. Socket Property:
    By default all input values are stored as socket properties, defined by the socket type (see #bNodeSocketType). This works out of the box without specifying further details of the socket, but has the disadvantage of requiring a very generic property definition. All sockets of the same type will use the same value ranges, subtype, etc.
  2. Node Property:
    Input values can be defined so that they use a property of the node they belong to. This has the advantage of allowing a detailed individual property definition, including value range (min/max limit), subtypes, units, etc.
    To use a node property as the input value of a socket, the value_property string of a socket must be set and a node property with this name must be defined as part of the node type.
    # Create an input socket called Factor
    # The socket will use a generic, unlimited float value by default!
    socket = node.inputs.new("NodeSocketFloat", "Factor")
    # Assuming "node" has a property "input_factor", use this as the input value insteaad
    socket.value_property = "input_factor"
    In addition to value_property the socket can have a value_path string which defines a relative path on the node.
    # Use a nested struct "settings" in node, resulting property used will be "node.settings.input_factor"
    socket.value_path = "settings"
    socket.value_property = "input_factor"

Node Groups

Node groups are node trees that can be linked from the library by other nodes. The Group node type stores a pointer to a bNodeTree as an "internal" node tree.

Terminology
In the following the terms "node group (tree)" and "group node" are used, which might be a bit confusing but is an important distinction: The "node group" refers to the bNodeTree data block from the library, while "group node" means an instance of the Group node type (a bNode) that links to such a tree data block.


When a node tree containing a group node is evaluated, the internal group tree replaces the group node. This makes it possible to create multiple instances of a set of nodes, be it in the same node tree or in different trees. Node groups can be used as predefined "assets" and be reused in new node setups without having to copy or recreate the nodes.

Node groups can not be used directly in integrated material, compositor or texture node trees. These data blocks create a static node tree on their own, which is not in the library itself. They can, however, include a group node which in turn links to such a node group tree.

See #NodeEditor_NodeGroups for how to edit node groups in the interface.

Interface

A group node can be connected to other nodes using the node group interface. The interface of a group node is defined as part of the group tree and automatically updated for all instances of that group, i.e. all group nodes linking to the same tree share the same interface. See also #NodeEditor_InterfaceDefinition for how to change this definition.

Inside the node group tree there are two special types of nodes: Group Input and Group Output. They mirror the sockets of the group node, so that internal nodes can be mapped to sockets of the group node instance.

DNA/RNA structure

Each bNodeTree can store a list of input/output interface sockets. The DNA struct used for this is also a bNodeSocket, but it should be noted that these are not regular node sockets. They merely store information about how the actual sockets of a group node instance and the internal group input/output nodes should be created.

DNA Compatibility
Using bNodeSocket as DNA structs for interface information is confusing, but makes it a lot easier to keep the node system both backward and forward compatible. Older Blender versions used the inputs/outputs lists in bNodeTree as actual sockets, to which real nodes could connect. This system had some serious flaws (awkward UI with link node links, non-local bounding box behavior, special case checking for fromnode/tonode pointers in bNodeLink). It has been replaced by dedicated GroupInput/GroupOutput nodes (see #GroupInputOutput).


Apart from the socket's name and identifier strings all interface information is stored in ID properties and depends on the data type of the socket. Data socket types usually store details of the input value properties here, such as default value, numerical range, subtype, etc.

Each bNodeTree with interface sockets defines an accompanying RNA type for socket data, based on the PropertyGroup type. The name of this RNA type is generated by adding the bNodeTree ID block name as a suffix to "bNodeTreeInterface_<name>". Group node instances then create a property group of this type named "interface" in which any socket input values can be stored. This allows node groups to register detailed properties for each socket, just like regular nodes can define properties for their sockets. Using a separate property group type is necessary to allow switching the tree pointer of a group node instance.

Nested Node Groups

It is possible to create group nodes inside a group tree, just like in any static node tree.

Compatibility
Older Blender versions did not allow group nodes inside other node groups at all. This was a design decision to simplify coding, but restricted the usefulness of node groups severely. Care has to be taken when using this feature though, as it can easily break forward compatibility and make it impossible to use such nested node groups in older Blender versions.


One important common limitation is that node groups can not be nested inside themselves. This is not dictated by the DNA structure itself though, but simply makes it impossible to "inline" node groups due to infinite recursion. Advanced future node systems may well allow a feature like this, but all current node systems need to prohibit such recursion. The group node's node_tree property has a poll function which checks for any recursion by doing a dependency search in the respective bNodeTree. If the parent node tree of a group node is already linked somewhere inside a group tree, this tree is not eligible for linking.