Transcript SIGG98.ppt

Synthesizing Realistic Facial
Expressions from Photographs
Frederic Pighin Jamie Hecker Dani Lischinski
David Salesin
Richard Szeliski *
University of Washington
v The Hebrew University
* Microsoft Research
v
Realistic facial animation
One of the most challenging problems in
computer graphics:
• Facial surface is subtle
• Facial movements are complex
• Faces familiar
Muscles
Bones
[Parke and Waters, 96]
Our approach
3 steps in synthesizing realistic faces:
1. Modeling from photographs
2. Creating new expressions
3. Animating expression changes
“surprised”
“sad”
“worried”
“neutral”
“joy”
Previous work: Modeling
• Hand digitizing
• 3D scanning
• Image-based techniques
Hand digitizing
[Kleiser, 95]
3D scanning
[Lee et al., 95]
Image-based techniques
[Parke, 74]
Previous work: Animation
• Performance-driven animation
• Physically-based animation
• 2D morphing
• 3D morphing
Performance-driven animation
[Williams, 90]
Physically-based animation
[Lee et al., 95]
2D morphing
[Beier and Neeley, 92]
[Bregler et al., 97]
3D morphing
[Chen et al., 95]
1. Modeling from photographs
Overview of modeling
1. Take multiple photographs of a person
2. Establish corresponding feature points
3. Recover 3D points and camera
parameters
4. Deform generic face model to fit points
5. Extract textures from photos
Animating expression changes
Varying the proportion of different
expressions over time creates animation
Pose recovery
Solve for:
• 3D locations of feature points
• Camera orientation
• Camera position
• Focal length
Robust estimation by linear least squares
Mesh deformation
Involves two steps:
• Compute displacement of feature points
• Apply scattered data interpolant
Generic model
Displacement
Deformed model
Texture map extraction
The color at each point is a weighted
combination of the colors in the photos
Texture can be:
• View-independent
• View-dependent
View-independent
View-dependent
2. Creating new expressions
Creating new expressions
In addition to global blending we can use:
• Regional blending
• Painterly interface
X
+
“neutral”
X
“happy”
“fake smile”
Design of a drunken smile
3. Animating expression changes