Technical Report MSC-2017-20

Title: A Deep Learning Perspective on the Origin of Facial Expressions
Authors: Ran Breuer
Supervisors: Ron Kimmel
PDFCurrently accessibly only within the Technion network
Abstract: Facial expressions play a significant role in human communication and behavior. Psychologists have long studied the relationship between facial expressions and emotions. Paul Ekman et al.(’78), devised the Facial Action Coding System (FACS) to taxonomize human facial expressions and model their behavior. FACS is an anatomically based system for describing all observable facial movements for each emotion.

The ability to recognize facial expressions automatically, enables novel applications in fields like human-computer interaction, social gaming, and psychological research. There has been a tremendously active research in this field, with several recent papers utilizing convolutional neural networks (CNN) for feature extraction and inference.

We employ CNN visualization and understanding methods to study the relation between the features these computational networks are using, Ekman's FACS and the Action Units (AU) that comprise it. We verify our findings on several datasets, among which the Extended Cohn-Kanade (CK+), NovaEmotions and FER2013 datasets. We apply these models to various tasks and tests using transfer learning (or knowledge transfer), including cross-dataset validation and cross-task performance. Finally, we exploit the nature of the FER based CNN models for the detection of micro-expressions and achieve state-of-the-art accuracy using a simple long-short-term-memory (LSTM) recurrent neural network (RNN).

CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF files directly. The latter URLs may change without notice.

To the list of the MSC technical reports of 2017
To the main CS technical reports page

Computer science department, Technion