Technical Report CIS-2004-01

Title: Representation Analysis and Synthesis of Lip Images Using Dimensionality Reduction
Authors: Michal Aharon and Ron Kimmel
Abstract: Understanding facial expressions in image sequences is an easy task for humans. Some of us are capable of lipreading, by interpreting the different motions of mouths. Automatic lipreading by a computer is a challenging task, with so far limited success. The inverse problem of synthesizing real looking lip movements is also highly non-trivial. Today, the technology to automatically generate an image series that imitates natural postures is far from perfect. We introduce a new framework for facial image representation, analysis, and synthesis (here we refer just to the lower half of the face with a focus on the mouth).It includes interpretation and classification of facial expressions and visual speech recognition, as well as a synthesis procedure of facial expressions that yields natural looking facial movements. Our facial image analysis and synthesis processes are based on a parameterization of the mouth configuration set of images. These images are represented as points on a two-dimensional flat manifold that enables us to efficiently define the pronunciation of each word and thereby analyze or synthesize the motion of the lips. We present some examples of automatic lips motion synthesis and lipreading, and propose a generalization of our solution to the problem of lipreading different subjects.
CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF files directly. The latter URLs may change without notice.

To the list of the CIS technical reports of 2004
To the main CS technical reports page

Computer science department, Technion