User:Aligorith/SummerOfCode2009

From BlenderWiki

Note: This is an archived version of the Blender Developer Wiki. The current and active wiki is available on wiki.blender.org.

NLA System for Animato

The aim of this project is to implement a new NLA System for Blender 2.5 using the new Animato animation system architecture. Most of the coding will take place during a 3 week period at the start of July as a Google Summer of Code Project in 2009.

Officially, this project aims to clarify what exactly NLA is supposed to do, streamlining it so that its core business is clear and simple to use, while also conveying this in a more coherent manner to the end-user. By doing so, it is hoped that this will enable more users to be able to harness the power of NLA for their workflows.

Quick link to progress notes: Progress Notes

Quick link to docs in progress (thanks Mikahl!): User Docs

Intended use scenarios

These fall into two broad categories:

Use of a set of predefined clips (i.e. actions from some motion library), to from a longer action (some apps call this a ‘choreography’)
Layering motions on top of each other
- E.g. 1.) Blocking pass + Refinement Pass 1 + Refinement Pass2 + etc.
- E.g. 2.) Basic walk cycle + Body Acting + Facial Acting + etc.

There is a considerable amount of overlap between these two usages. For example, in the second layering example, the basic walk cycle may be defined by a set of predefined clips. However, when thinking about how NLA systems should work, it is useful to think in terms of these two aspects separately.

It should be remembered that NLA should be suitable for Character Animation purposes. On primary characters, there is more of a focus on high quality motions, while on secondary characters, ease of use is more important. However, non-character based animation should not be neglected as per the old system, since a set of usable tools for those aspects is also vital.

Scope of NLA

Using the above usage categories, the two most obvious functions of the NLA should be apparent:

Ability to use/re-use Actions, and arrange them in some sort of chronological order – range of frames from Action to use, playback speed of Action, cycling of Action, etc. Actions referenced in such a way will herein be referred to as ‘clips’.
Ability to control the way in which clips are combined – influence (the equivalent of ‘alpha’ for materials), transitions, and blending methods

As Actions can be used to animate all logically animateable properties (i.e. all except rendering properties, and a few other ‘system’ settings) the NLA should be available for all data types that Actions are available for. In other words, the NLA system should work everywhere (“everything is animateable!”).

However, there are quite a few additional issues that the old NLA system covered which should NOT be included under the NLA umbrella (at least directly by the built-in tools):

The most prominent of these, is the so-called “Auto-Walker” functionality. These consist of a ‘rig-level’ part and a ‘motion-control’ part. Rig-level issues for these are things like walks following paths, and non-slip footsteps. The motion control issues are simply “what cycle to use”, which is the focus of the NLA system.
Game Prototyping and Crowd Simulations aren’t primary goals that the NLA should be concerned with. The primary difference here is that these are ‘state-based’, instead of ‘time-based’.

UI Design Requirements

In addition to the aforementioned issues, a number of basic issues should be noted so that they are not forgotten:

NLA Editor is strictly for editing clips/strips only. No keyframes. While we lose a bit of flexibility in lining up keyframes and strips by doing this, having both in the same editor at the same time was often a burden for users (context was muddled) and also in the code (checks for keyframes vs strips needed to be performed, and keyframe editing code needed to be duplicated here). Keyframe editing really belongs to the within Action context, and we already have two editors for that already (plus the DopeSheet practically rules out the traditional arguments for keeping keyframes in the NLA Editor previously). NLA however is on a higher level than keyframes, working with Actions as single units (like blocks).
Users should not have to worry about NLA when keyframing normally. Even when they have an NLA setup, the workflow should be practically the same as without any NLA, with none of the associated bugs...
Multiple NLA Strips can exist in a single row (i.e. a ‘track’ can have multiple strips) but these must not overlap in any way.
More modern OpenGL tricks for drawing strips. The old ways were really dire.
Evaluation order should be presented in a 'bottom up' fashion to better match mental models of 'layers' as per 2D paint applications such as GIMP and Photoshop.
Investigate the value/worth of having 'meta-strips' and other related forms of encapsulation.

Motion Mixing

Motion mixing could be considered the primary function of NLA. Following through with our two-category description of the functions of NLA, two types of mixing needs to occur:

Horizontally (i.e. time-wise, a -> b) – one strip to the next one, usually within the same track
Vertically (i.e. layering/mixing, a + b) – one track on top of another

Transitions:

Concerns what happens as time progresses between strips
In general, the only “horizontal” mixing that goes on is with some kind of “auto-inbetween” (or “transition”) clip between two adjacent clips. This should only occur if the user explicitly demands it. Only channels common to both clips would get blended, with the other tracks obeying standard mixing procedures.
By default, clips should keep holding on until the next one in that track is encountered. The first clip in the track would extend backwards (i.e. to frames before it too).
Clips could be made to not hold, in which case, there would be no contribution from that track until another clip was encountered. However, if no other clips elsewhere override the properties affected by the clip, then there would be no net effect.

Layering/Vertical Mixing:

Tracks act as a form of animation layer
Blending (i.e. varying contributions from both clips, which can be controlled via F-Curves instead of just having blend-in/out frame ranges) – this is the existing default… does it work well enough though?
Add on top of existing channels – apparently this existing option doesn’t work well? Examples?
Subtract?
Multiply?
This is represented in the editor as bottom-up evaluation order, since this more closely resembles the layering/stacking concept.

Fate of ‘Orphaned’ Features

Auto-walking/stride/etc. features are more of a rig-level feature than an NLA feature. The old NLA-modifiers and stride offsets, etc. were really being evaluated as part of the pose system (but before constraints) instead, so it is more logical to move those there in line with constraints.
NLA-Modifiers
- as mentioned, only one of these was ever implemented (Curve-Deform) and it only worked on bones, so that one could be moved to the pose-system, perhaps with influence controls to turn it on/off over time.
- the functionality of the unimplemented ones is very similar to that of some of the F-Curve modifiers. Therefore, F-Curve modifiers will be able to be applied to all the F-Curves of a given strip, by defining modifiers for that strip.

Proposed NLA System (based on original design from 1.5-2 years ago now)

(see also http://lists.blender.org/mailman/listinfo/bf-animsys for discussions on this)

NLA data resides in AnimData, and is available wherever AnimData is.

NLA data consists of a set of ‘tracks’ which could be considered as “poor-man’s animation layers” in which multiple ‘strips’ (or should we call these ‘clips’) are found. At any one point along a track, only a single strip can occur, or in other words, strips cannot overlap.

An AnimData block’s action is always evaluated after NLA evaluation for that AnimData has been completed. Therefore, this is always presented as being the last track of a set of NLA tracks. Now, since the last track is always the AnimData’s action, which is the action that can be edited in the (Keyframe) Animation Editors (i.e. DopeSheet and Graph Editor), it follows that keyframe editing of strips used in the NLA can only be done on the last or ‘tweaking’ track.

This is particularly good for one of the usage scenarios mentioned at the start, the “BlockingPass + RefinementPass1 + RefinementPass2 + etc.” approach to NLA usage, allowing it to emulate the functionality of ‘Animation Layers’ quite well. However, what happens when the user wants edit some action in some track other than the last one (i.e. in the scenario considered here, what if they want to make some change to one of the refinement passes)?

To edit some referenced action in some earlier track, the user simply needs to select the strip in question and “enter editmode” (i.e. TAB key) on it as per the conventional Blender workflow. Several things happen at this point: 1. All tracks after the track the user clicked in (including the existing ‘tweaking’ track) are muted and are greyed out (we they probably shouldn’t disappear completely, otherwise the user would think they’ve just lost most of their NLA animations). This is quite consistent with many construction history stacks used in modelling systems, and helps the user to be able to focus on what exactly the strip in question contributes.

2. A new tweaking track appears immediately after the track the user clicked on, referencing the ‘active’ strip (i.e. the one the user clicked on) in that track. This temporarily becomes the AnimData’s new active-action. As usual, the new tweaking track is the last one in the NLA stack which gets evaluated. Editing of keyframes can now proceed as usual. Playback/evaluation of the NLA system will only include the tracks up to the current tweaking track. When the user is done editing this action, they can simply “exit editmode” (i.e. TAB key), and the NLA is restored back to ‘normal’ but with the changes applied.

What happens though if the action being edited in such a way is referenced more than once (i.e. another strip in another track - perhaps for the sake of argument, in the track above - also references it)? As always, we try to present a consistent interface to the user. Therefore, this situation works EXACTLY the same as the way described above. That is, the tweaking track only appears for the track+strip the user specifically requested, nothing more. As this editing could potentially have unintended for the user (i.e. they only wanted the tweaks to be applied to that specific strip), we can also highlight (in dull red for warning?) the other strips which also reference the same action. However, at no point should we try and guess what the user wants, and auto-create a new copy of the action for that strip. Rather, we provide ways for the user to rectify the situation instead.

One situation where the best course of action is unsure though, is what to do when in Edit Mode, the user wants to switch to editing another strip/action from the same track. Code-wise, it’d be much easier to just require the user to exit Edit Mode, select the next strip and enter Edit Mode again. However, it is very likely that this will be too much of a cumbersome workflow for users, and being able to switch to another action within the same track might be a more viable option.