My goal was to create a facial rig that could be driven by an alternative animation source such as motion capture and be easy for animators to use. In this case, I chose to use FaceFX, a facial animation solution I have used in the past on the Mercenaries franchise, but this rig could easily take motion capture. In fact I initially used motion captured from a Kinect.
The rigging is made up of three parts: skeleton, blend shapes, and controls. The skeleton is a shadow that is kept completely separate from the control rig. This is an approach I’ve used in all my production rigs. It allows the underlying skeleton to change without compromising the rigging. Control of the skeleton is handled through a standard rig with a simple blend shape mask that drives the controls.
For the blend shapes, I used basic phonemes and expressions. The base shape has nulls riveted to the vertices, which influence the hand animation controls but leave them free for animator tweaking. This influence can be blended in and out via an animatable parameter. I wanted the blend shape mask to be generic for use on other characters. Also, the mask vertices could line up with motion capture markers allowing the data to drive the control rig and would be easy to animate over.
Here is a video of the results. I used a FaceFX sample animation and audio clip to drive the blend shapes. The animation is in an XML format. So I wrote a plugin for Maya that parsed the file, turning its data into animation curves on the blend shape.