|
Expression-invariant 3D face recognition
A visual experiment showing that for the human eye, 2D information plays a more important role in the recognition of a face than 3D information, though such information can be misleading
(the faces of Bush and Bin Laden in this example are just drawn on the 3D facial surface of the same person).
Three-dimensional face recognition is an emerging modality, trying to use 3D geometry of the face for accurate identification of the subject. While traditional two-dimensional face recognition methods suffer from sensitivity to external factors, such as illumination, head pose, and are also sensitive to the use of cosmetics, 3D methods appear to be more robust to these factors. Yet, the problem of facial expressions is a major issue in 3D face recognition, since the geometry of the face significantly changes as the result of facial expressions.
We developed an expression-invariant 3D face recognition approach based on the isometric model of facial expressions. According to this model, a person's identity is associated with the intrinsic geometry of his or her facial surface, while the facial expressions are associated with the extrinsic geometry. Using this model, the problem of expression-invariant face recogntion boils down to comparison of nearly-isometric non-rigid surfaces.
It is possible to represent the intrinsic geometry of the facial surface by embedding it into a low-dimensional Euclidean space with minimum metric distortion. The embedding is performed using Multidimensional Scaling (MDS). As the result, an expression-invariant representation of the face called canonical form is obtained. MDS undoes the deformations of the face and allows to treat the canonical form as a rigid object.
Papers
A. M. Bronstein, M. M. Bronstein, R. Kimmel,
"Expression-invariant representation of faces", IEEE Trans. Image Processing, Vol. 16/1, pp. 188-197, January 2007.
A. M. Bronstein, M. M. Bronstein, R. Kimmel,
"Robust expression-invariant face recognition from partially missing data",
Proc. European Conf. on Computer Vision (ECCV), pp. 396-408, 2006.
A. M. Bronstein, M. M. Bronstein, R. Kimmel,
"Expression-invariant face recognition via spherical embedding",
Proc. Intl. Conf. on Image Processing (ICIP), Vol. 3, pp. 756-759, 2005.
A. M. Bronstein, M. M. Bronstein, R. Kimmel,
"Three-dimensional face recognition",
Intl. Journal of Computer Vision (IJCV), Vol. 64/1, pp. 5-30, August 2005.
A. M. Bronstein, M. M. Bronstein, R. Kimmel,
"Isometric embedding of facial surfaces into $S^3$",
Proc. Intl. Conf. on Scale Space and PDE Methods in Computer Vision, pp. 622-631, 2005.
A. M. Bronstein, M. M. Bronstein, E. Gordon, R. Kimmel,
"Fusion of 2D and 3D data in three-dimensional face recognition",
Proc. Intl. Conf. on Image Processing (ICIP), pp. 87-90, 2004.
A. M. Bronstein, M. M. Bronstein, R. Kimmel, A. Spira,
"Face recognition from facial surface metric",
Proc. European Conf. on Computer Vision (ECCV), pp. 225-237, 2004.
A. M. Bronstein, M. M. Bronstein, R. Kimmel,
"Expression-invariant 3D face recognition",
Proc. Audio- and Video-based Biometric Person Authentication (AVBPA), Lecture Notes in Comp. Science No. 2688, Springer, pp. 62-69, 2003.
Patents
M. Bronstein, A. Bronstein, R. Kimmel,
"Facial recognition and the open mouth problem", European Patent No. EP1849122, October 2007; US Patent application No. 20050180613, August 2005.
A. Bronstein, M. Bronstein, R. Kimmel,
"Three-dimensional face recognition", European Patent No. EP1550082, July 2005; US Patent No. US6947579, September 2005; PCT No. WO2004032061, March 2004.
See also
Non-rigid shape similarity and correspondence
Partial similarity and correspondence
3D face recognition in CNN news
Technion Excellence Program
|