Doc:2.4/Tutorials/Animation/BSoD/Character Animation/Sound and lip sync

From BlenderWiki

Adding sound

When learning computer programming, tradition holds that the first program you write displays "Hello, world!" on the screen. In keeping with that tradition, here is a sound file of someone saying "Hello, world!". It has been converted to a 16-bit sound file in the free audio software Audacity.

The sound file

The following file is actually a .wav sound file, save it to disk. Media:HelloWorld-16bit.wav

Change to the Video Sequence Editor.

Change one of the windows to the Video Sequence Editor. I changed the NLA Editor into a Sequence Editor.

Add a sound file.

In the Sequence Editor, choose Add>>Audio(Wave) from the menu.
Load the HelloWord-16bit.wav file in the file browser window.

The audio comes in to Blender as a strip, automatically entered into grab mode. Numbers at the beginning and end of the strip show the frame numbers that it starts and ends. Move the mouse so the sound starts at frame 1. You can trim the ends of the sound strip by right clicking on either end, and you can move the sound strip by right-clicking on the center of the strip.

The new sound file. Note numbers indicating the start and end frames (move it to start at frame 1).

The sound buttons.

Change the Sequence window into a Buttons window.
Click the Scene context button, and then the Sound block button to access the sound settings.

Enable both Sync and Scrub.

In the Sequencer panel, select both Sync and Scrub. Sync will make the animation follow the audio, even if it has to not draw some frames to catch up. Scrub allows us to drag the frame slider and hear the audio while doing so.
Move the frame slider back and forth in the Timeline window: you can hear the audio as you do so, no matter how slowly you drag the slider.

Here's the plan: we will animate the facial expressions to say "Hello, world!", then convert that into an action. Then we will be free to move the sound file, along with the lip sync action, anywhere we want in the animation.

First, select the armature.
Make sure the Rest Position button is still pressed in the Armature panel, to keep the armature from moving while we animate the lip sync.
Now select the character mesh.

To start, set all shape keys to zero.

In frame 1, add a key frame of 0.0 for all shape keys by grabbing the sliders and moving them up and then back again to 0. This gives us the beginning boundary of the lip sync.
Advance the frame slider to find what frame the sound ends (at about frame 22). Add a 0.0 key for all shape keys here, to set an ending boundary. Now we can easily see that all lip sync animation has to occur between these two sets of keyframes.

Now comes the part where you start muttering to yourself and looking in the mirror to see how your mouth moves. Our first pass will be on setting the Open keys. When I say "Hello, world!", my mouth opens a little on "Hel", it opens more on "lo", it closes all the way on "w", opens a little on "orld", and closes after the "d" is formed.

Move the frame slider to listen to the audio and find where the "Hel" is. I set the Open shape to 0.39 at frame 2.

Note

Whenever you add a keyframe, it is added at the position of the frame slider, which is always exactly on a frame. However, once the keyframe is added, you can grab it and move it in between frames.

For the Open shape, I set keys as follows: