From BlenderWiki

[edit] Tutorial Part 1

Ok, so let's work through a practical example. In this example, as an end result, we are making a really excellent CG movie, and one shot calls for our character to be standing still, and he shakes hands with the protagonist who walks up to him.

Concept: Many months ago, a MoCap professor at a university dreamed up filming a handshake because they were studying complex human interactions as part of a psychology grant. Motion Capture: Justin and Rory at Carnegie Mellon donned the suits and shook hands. Digitization: Using the ball setup shown above, the video was digitized into a C3D file format.

For this tutorial, we assume we want our character to shake hands with another character. This is actually a very complex motion, involving lots of subtle movements and gestures, so we want to use a MoCap to aid us in developing a realistic animation.

We sift through the C3D and CM catalogs, and find [Subject 18 Session 02] where two characters take a step forward, shake and release. Download the video (mpg or avi file) and watch it to see the actors/animation in action. Observe that the handshake is fast and firm, a brief meeting, a bit stiff and formal, where the actors are not going to stand and chat. To complete this tutorial, visit their site and download the linked C3D file.

	Library:
	I keep my MoCap files in my C:/Blender/Lib/Pose folder, named by <taxonomy> <Source> <Session>.c3D, so the linked file is named "team-hand-shake CMU 18_02.c3d" when saved on my disk.

[edit] Workflow

So, drilling down into our process, after you have selected and downloaded a MoCap file, you need to:

1. . Install the Import and Baking Scripts
2. . Import the MoCap file into Blender and parse the data into framed Empty positions
3. . Select/Create your armature and Constrain bone positions to empties
4. . Edit empty Ipo curves to remove stutter
5. . Fill in missing targets and bone movements
6. . Record as NLA Action
7. . Integrate Action into character motion/action sequence

[edit] Installing Blender Extensions (Scripts)

Blender provides a Python API for extensions, and three major tools were written to process Motion Capture files:

1. . Import BVH files, and make an armature which tracks those markers
2. . Import C3D files, and make an armature which tracks those markers
3. . Bake either armature motion into a scalable, pose-able armature

Click here to get the zip file containing the latest C3D import script, then unzip the script from the file into either:

• your Blender Scripts directory, and use it to replace the scrip that is there, or

The Blender Scripts directory is specified in the User Preferences window->File Paths tab. If you put the file in your Blender Scripts directory specified in your User Preferences, and it replaced the existing one, you will be able to run the file through the Blender menu system: File->Import->Motion Capture (.c3d)....

• a folder of your choice.

If you put the file somewhere else, select the Scripting screen layout, and in the Text Editor window, select File->Open and navigate to that file. After it is loaded, select File->Run Python Script or press Alt P while your mouse is over the window.

Click here to get the zip file containing the latest Bake Constraints import script, and save the .py file as described for the C3D import script.

[edit] Importing a MoCap file

Run this file as described above. One of your windows will change to a file browser. Navigate to the Shake Hands Mocap file that you downloaded previously, and click the Import C3D button. Bring up your console window and watch it fly. While the program is not guaranteed to process all C3D files, it can produce, for most files:

1. . An cloud of empties that represent each marker
2. . An armature if the marker set can be recognized. The marker sets supported are HumanRT.mkr and HumanRTKm.mkr.

The program will process the C3D file as best it can, and abort if too many errors are found. When attempting to process files, you may wish to adjust (some programming skill is required):

DEBUG: this constant controls the amount of information sent to the console about the status of the program. When tracking down errors and examining data inside the file, set this to a higher level. Valid range is 0 to 100.

The amount of time to process the file depends on the amount of data in the file, and the available capability of your desktop. For the longer portions of file processing, the program will display a message giving you an estimate of the time required to process the data set for that stage. That estimate will vary from actual because of increasing memory needs for processing larger files.

[edit] Reviewing the Cloud

When the import script has completed processing our Team-Hand-Shake test file, your display will be positioned at frame 317, and your Frs/Sec will be 120. Your 3D View will have about 40 empties forming two clouds, and two armatures inside each cloud. at location 0,0,0, all neatly stacked on one another. To look at the first frame of the motion capture, press the right arrow to advance to frame 318. If you zoom out, you will see the empties form a cloud of data points, occupying a space from (-60,40,0) to (40,20,80)

	Actors can be Giants:
	To see the empties and armature, you may have to scale view outward via middle mouse wheel or the keypad -. You may also have to set your View properties Clip End to 10000, since some MoCap is scaled for maximum resolution.

Press Alt A to see the handshake. As you can see, every empty is moving for every frame. There's a lot going on there! These 300 frames of animation represent less than three seconds of real-time motion. If anything, you now appreciate the job of an animator.

Now is a good time to Save.

Each empty corresponds to a marker. So where were the markers? Well, thankfully, Carnegie Mellon has a method to their madness and provides you [pictures of the actor and their suit]. They also have named the markers in a cryptic short hand; for example, RKNE is the right knee, and LSHO is the Left Shoulder, etc. The import script has used these names, and has named these empties what they were named in the file. For this file, since there were two subjects (each wearing a suit), the marker names are prefixed with their names, Justin and Rory. The script therefore created two armature and objects named Justin and Rory, and prefixed the empties with those names. Using this information, the script was able to construct armatures that are constrained to move as the empties move. There may be other markers in the file, which are redundant or not necessary to achieve a realistic motion.

Your playback speed will vary based on the power of your computer and number of markers in the file. Real-time playback is only accomplished by rendering a video file.

[edit] Pre-visualizing the Cloud and Armature

"Pre-vis" is a technique just to see how the motion blocks out and whether the animation is any good or will suit your needs. Pre-vis is a quality check and a quick way to preview the animation and decide whether to keep or discard an animation. Blender uses the OpenGL library to render previews, either dynamic or rendered. Normally, empties and armatures and not rendered as part of an animation; they are "invisible" to normal rendering algorithms since they have no "faces" or material. However, Blender provides two ways to pre-vis animations:

• The dynamic preview is invoked via Alt A but is not real-time, often slower than real-time. This gives you a chance to carefully watch the motion, and identify areas that are good, and which are not.
• Rendering is obtained by Ctrl LMB clicking the Render button/icon located on the far right of the 3D View header. Doing so will create an animation file (avi or mov), and in the size specified in your Render Format settings.

Blender provides 20 layers as a way of organizing your work. An object is located on one or more layers, and displayed in the 3D view when one of its layers is selected. The Import Script is programmed to put the Empties (markers) on layers 1 and 12, and the Armature on layers 1 and 2. Thus, selecting layer 1 shows both the cloud and the armature; selecting layer 2 shows only the armature, and selecting layer 12 shows only the cloud. The button for layer 12 is located just beneath (as a foundation) of the layer 2 button.

If you select only layer 12, you will see the original cloud of markers as they were captured in the lab. When dealing with an unknown marker set, you will need to pre-vis the cloud in order to manually determine and construct an armature to drive any mesh character or avatar.

The video to the right shows the Armature only (layer 2) pre-vis of the Play-Climb-Jump mocap session from CMU file 01_13.c3d. Watching this pre-vis, we have a fairly representative example of a poor motion capture session. He makes two cycles of (ducks under a low bar to the right, duck under to the left, climbs and jumps down to a landing). At various times the system loses track of some markers, but then recovers.

Switching to the Animation screen layout, we use the Timeline window type to isolate those frames that have useful information. This layout features the Timeline window, which, with its VCR-style controls, makes it easy to examine and scrub through animations. There is one point at which he is in the air, fully stretched out, and lands, which might be useful in a library. We examine the file and determine that frames 1300-1425 and 2500-2650 capture this motion. We mark those ranges for future study.

[edit] Cleanse Data

Data cleaning, or cleansing, is the act of examining data and editing or removing bad data. In MoCap, this will show up as a marker that suddenly jumps out of place, jitters, or stays "stuck" somewhere in 3D space. The import program has an option help you with cleaning, by setting the constant CLEAN true, the program will ignor markers that are very close to origin or that are beyond 10K BU. If the program was able to automatically made a constrained rig, the bones that are constrained by those markers will suddenly snap out of place. The good news is that once you clean up the marker Ipos, the constrained bones will behave.

[edit] Remove Discontinuity

Working with this same file, play-climb-jump CMU 01_13.c3d, (assuming you turned CLEAN to False) you will see a good example of this. In frame 1913-1950, notice marker LBWT, his Left Back Waist marker as shown in the figure to the right. You can select this marker using the outliner, and then zooming and positioning the Ipo window to show this frame range, as shown in the image to the right (mouse-wheel down to zoom out to see the curves). This frame range is as he is ducking under the bar, and the cameras lose track of this waist markers, and it jumps. In looking at the black Ipo curves (which are actually black because of all the data points!), you see how there is almost a break in the line and vertical yellow and orange bars, as the curve drops to 0. As a result, his waist, root, and leg bones bones snap out of place, snapping him into a completely different pose in a millisecond. As he continues to move though, the markers clear the obstruction and are detected by the tracking cameras; at frame 1951 they then resume their normal placement.

Thankfully, the import script has created a continuous bezier curve that connects a marker's position from frame to frame. Simply delete the errant locations and blender will interpolate a smooth motion and transition of that marker during the missing frames. To do so, click on the LocX label in the Ipo window, and the display will only show that curve.  RMB click on the curve to select it, and press Tab to enter edit mode. The curve will turn from black to pink. Press B and drag a box around those bad boys that jump to zero. When you let up on the mouse button, they will turn yellow as shown to the left. Press X to delete them, and the curve should now look like the image to the right. Because all points are connected in an interpolation curve, once the bad points are removed, Blender can interpolate smooth motion between the two known good points.

Now repeat cleaning the LocY and LocZ curves for this marker. For LocY, you may want to scale the Ipo display up in the height (Y) direction by {CMB|M}} and moving your mouse up. This scales the display window higher, allowing you to clearly see where the marker jump down to zero and then back up again. Repeat the edit-box select-delete action. If, on entereing edit mode, the curve turns yellow (all selected), simply press A to un-select all the datapoints. After a few markers, you fingers will know to do tab-a-b-x-tab. As a matter of practice, I like to leave an Ipo curve NOT in edit mode (not pink, but white or black or its normal channel color). Shift LMB click all the channels to see all three curves back in action.

Repeat this for the lost head marker RBHD, where it is easy to spot the jump to zero.

I have noticed that some MoCap files create a <name>"-1" marker, e.g. "RANK-1", when a marker is lost. Looking for these named empties is a quick way to see if a marker was lost for some time.

[edit] Remove Jitter

The RBHD marker also has a glitch, where the cameras got confused between it and an arm marker passing near it between frames 1903-1908. To cure this, simply delete the LocX, LocY, and LocZ data points to remove this jitter.

Congratulations! Just cleaning up those two markers has salvaged the motion from 1730-1970.

[edit] Build Your Rig

I have hard-coded the ability to create a standardized armature based on known marker names in a C3D MoCap file. In testing the file, I ran across C3D files which used different marker sets; for example, one file was used to study the shaking that accompanies Parkinson's disease. For these cases, you will have to build your own rig. If the import script cannot recognize the marker set, it will tell you so in the console and give you the name of the marker set. You can then contact the C3D file encoder for images or information about the marker placement, and build your own rig. This section of the tutorial walks you through the process of building your own constrained rig for these circumstances.

[edit] Where to Start

First, let's assume you have some sort of skeleton driving the character, but for some reason the script could not recognize the markers and the encoder simply named them "em_xx" where xx is just a number. This is worst case, since the names don't give you a clue. You contact the encoder, and they give you an image of roughly where the markers were placed. Now you have to just look at the cloud, watch it move through the animation a few times, from different angles, and begin to visualize the character that was marked. If you run the original C3D Import script (version 0.1) against the Interact-Shake_Hands file, you will get this cloud.

All animals have a hip, even snakes. That is a good place to start. I like to start with a root bone, a bone that is almost like a grabber, that starts off the whole chain. Moving just that one bone moves the whole armature. This bone can also serve as the stride bone (more on that later). You can study the armature created for the names of the bones and their constraints. For the purpose of this tutorial, let's start with the main spine bone and build up a chain of a few bones so you get the idea.

If you are using the Shake Hands file, you have locate em_20 and 21. These correspond to markers that were located to the right and left back waist. Between these is a good spot to start the spine bone, offset into the body a bit by the two front waist markers, RFWT and LFWT, which we have imported as em_2 and em_54. Go ahead and rename those empties to their proper name (type it into the Transform Properties panel or the Editing Links panel), and show their Name and Bounds in the 3D view (Object Object buttons, Draw panel). For this session and actor, press control+numpad-3 to get the Left view. This is a view of the actor facing you, and it makes most sense to start building the armature in this view. In practice, I use a 4-view workspace/desktop and really like the vertical panel arrangement for the buttons window.

[edit] Build and Constrain the Spine

Click your cursor somewhere in his body about waist high and Add->Armature. Out of habit, I immediately tab out of edit mode and clear any rotations of the object, then tab back into edit mode. Grab the translator and drag it up, or G-Z and move your mouse up to scale up the bone into his shoulder area. Tab out of edit mode into Object mode. Enable the draw of the Name and Axis. In the Editing context, on the Armature panel, enable Envelope instead of Octahedron, because envelopes are cooler. Also enable the display of bone axis (Draw Axes).

Now, in Object mode, add four constraints as described above, constraining the origin/location of this armature to a spot between the four empties. Note that I give a bigger weight to the rear markers than the front, because that is where your spine is. I used the full XYZ coordinates of each empty, his spine would drop if he bends over, which is not what happens. So, I disabled the Z for the front markers. For the Y values, I really just want to keep the spine centered in the geographic center. For the X value, that is what I want to modulate a little, so I add in a little of the X from the front markers. Little things like disabling the Z can add up to saving a lot of compute time.

[edit] Point the Spine

em_24 appears to be T10, possibly referring to the tenth lumbar vertebrae in the spine. em_25 is RBAC, and em_23 is C7, right at the base of the neck, em_63 is CLAV Clavicle, and below it, em_76 is STRN (Sternum)

Create a Floating Empty based on T10 and CLAV, positioned so we can point the tip of the spine bone. Use the Z of T10, but average of X and Y. Amazingly, when you Add->Empty, the Empty name is "Empty". Rename it to "ik_T10". Here's a tip: Change the spine bone to a Stick, and extend it all the way up to C7. Then adjust the Influence of the CLAV on the empty ik_T10 until it intersects that straight line. An influence of 0.365 Makes it cross exactly. Then edit the bone until its tip is at the empty and rename the bone to Spine.Lower. In Pose Mode, add an "IK Solver" constraint to the bone, chain length 1, to point to ik_T10. the bone should turn yellow, and the Constraint should not be red. I also decrease the number of iterations to 5 to save compute time.

[edit] Align the Spine

I like my armature spines to point up in the Z (blue) direction when standing up, Y (green) facing backward toward the rear of the character, and X (red) facing to his left. Just remember R-G-B=X-Y-Z, and use the Local translator widget if you ever get confused. Namely, the normal orientation for something in Front view, for example, a non-rotated cube added in Top view (no rotations). Select the Armature in Object mode, and clear any rotations and orient it with the base at that averaged location and the tip near and pointing to ik_T10. Now we need to track the spine so that as he turns his body, the whole armature will turn as well. This way, in the future, we will be keying all the bone movements as relative rotations to the core orientation of the armature. This way we have a rig where he can shake hands facing left, and just as easily shake hands facing North. All we have to do is spin the armature, and all the local motions within the armature stay realistic.

In Object Mode with the Armature selected, add two "Track To" constraints, one to RBWT and the other to LBWT. Notice that the defaults are just like I like them; the "To:" axis is Y and Up: is Z. You should now be able to run your animation. Observe that the armature stays geographically centered in top view. I like to select the armature, shift-select the four waist markers, and share them on a layer, like layer 6, by pressing M and shift-clicking on the Layer 6 button. Then when I get confused, I can view only layer 6 and see only the markers of interest (and the armature).

[edit] Extend the Spine

Let's float an empty somewhere in his neck, close to C7, and point a bone "Spine.Upper" to it. Now we begin a cycle workflow:

1. . Select the next set of markers to use and name them.
2. . Edit the armature to extrude the bone past where it needs to be, positioned as per other parts of the anatomy, almost like a ray (use the Stick display mode)
3. . Name the bone
4. . Float an empty between those markers to serve as the IK target
5. . Edit the armature to end the bone near the IK target
6. . Point the bone to the IK target via the IK Solver constraint
7. . Align the bone to mimic body twists via the Track To constraint

1. We will use the next set of markers as a guide: em_28 is RBHD, em_27 is LBHD, em_17 is LFHD, and em_11 is RFHD. Again these names are shown in the reference images on the Carnegie Mellon MoCap website. These form the headband.

2. Select the Armature and tab into edit mode. Select the tip and Extrude a bone segment by selecting the tip, E and moving your cursor up. This time, extrude the bone right up to the top of the head where the four head markers are, and put it right in the center of them. This reflects how your spine actually goes, as your skull is centrally pivoted and balanced on your spine - remarkably efficient, wouldn't you say? You will see that at frame 318 he had his head cocked to the left (a hint of recognition?) and tilted forward (eager?). Leave edit mode.

3. Call the bone Spine.Upper by tabbing into either the Transform Bone field, or the Armature Bones panel BO: field, and editing.

4. Add->Empty, and rename it ik_C7. Add two Location constraints, the first to C7 and the second to CLAV. I gave thh CLAV influence of 0.25, which floats the empty a little more toward the back of the neck, again only using the X and Y of CLAV, and not the Z.

5. Select the Armature, tab into edit mode, and drop the bone tip down to just below the empty ik_C7.

6. Go into Pose mode, Add an IK Solver as shown.

7. The base of the bone should mimic body twists indicated by the markers around the root of the bone. For Bone Spine.Upper, this is the empty T10.

[edit] Make the Neck and Head Bone

[Here is a training video of these next two steps].

For the Neck bone:

1. . Extrude the Spine.Upper upward and name the bone Neck.
2. . Float the target empty midway between all four head markers, but positioned more toward the rear markers.
3. . Point the neck at the empty
4. . Track the neck bone to C7

For the head bone:

1. . Extrude the Neck pointing forward toward the character's nose.
2. . Float an empty between the two front markers, named ik_Head.
3. . Point the head bone at ik_Head
4. . Float an empty between the two left side markers, named tt_Head
5. . Track the bone to the left middle marker.

Review your animation rig. Not a lot of head movement, but subtle. His head is tilted to one side, but he is looking the other way. I think this is a genuflecting type of gesture. You cock your head to the side in recognition of someone, and looking to the side is less confrontational.

[edit] Make Rest of Skeleton

Select the tip of the upper spine bone and extude a shoulder bone. From that, the upper arm bone is connected to the shoulder bone, the lower arm bone is connected to the upper arm bone, the hand bone is connected to the lower arm bone, and we all fall down (or however that song goes). For the legs, create a hip-upperleg-lowerleg-foot-toes set of bones, using the appropriate markers.

As you make the edit bones for the armature, keep notes on what empties you are using to locate the tip (tail) and head (if it has no parent). Once you have made one side of the skeleton, in edit mode, select the bones that make up the other side (all the non-spinal bones) such as the shoulder and arm bones, and use the handy Armature->Scripts->Armature symmetry script, and the other menu options there to auto-name the bones or change the .L to .R for you.

[edit] Constrain Your Rig

If you have just built your rig from scratch using the above steps, congratulations! You can now use any MoCap file in the following manner. Make the bone point via the IK Solver constraint, and to orient (Track To) toward certain markers. Create additional empties as needed to float between markers "inside" the body and constrain their location based on the outside markers. To keep tracked bones from rolling inexplicably, you may want to add a Limit Rotation constraint. If you keep good notes, you should be able to make a simple Node tree to create this armature reliably every time that marker set is encountered (see the module createHumanCMU for an example).

Ideally, future research will enable me to make an XML parser and allow you to just write an XML document to describe a marker set. As an interim, save your armature in a file, and then use the Append or Link function to roll in your armature into a new file; depending on the difference in size of the old/new character, the armature should work (if the markers are named the same from file to file).

1. . Import the MoCap file
2. . Each of the empties has an Ipo Curve.
3. . For each empty that constrains your rig, select it, and then select the new Ipo Curve in the Ipo Window

The empty that constrains your rig, for example RSHO, which used to use an Ipo such as Ipoem_23, will now use Ipoem_120, where Ipoem_120 is that marker, but from the new motion. As that empty moves along according to the Ipo curve, the bone will follow it, since the bone is constrained to it by your constraint rule.