Toward an Efficient Emotion Recognition from Facial Expressions Using ML

EasyChair Preprint 9929, version 2

Abstract

The information conveyed by facial expressions can be utilized to identify emotions. When these face expressions are performed, they change over time. Even for people, recognizing certain emotions is a difficult task. Machine learning algorithms are used in this work to recognize emotions in image sequences. It analyzes emotions automatically using cutting-edge deep learning on collected data. This paper compares current state-of-the-art learning algorithms for handling spatiotemporal data and adapting traditional static approaches to deal with image sequences. Expanded versions of CNN, 3D CNN, and Recurrent methods for universal emotion recognition are assessed and contrasted in two public datasets, where the performances are proved, and advantages and disadvantages are addressed. Afterwards, we propose a new end-to-end architecture called Spatio-Temporal Convolutional Features with nested LSTMs (Long short-term memory) that learns multi-level appearance features and temporal dynamics of facial expressions in a common way. More specifically, we use 3D CNN to extract spatio-temporal convolutional features from image sequences representing facial expressions, and the dynamics of facial expressions are actually combined by two sub-LSTMs, Temp-LSTM and Conv-LSTM modeled by a nested LSTM. That is, we use Temp-LSTM to model the temporal dynamics of spatio-temporal features in each convolutional layer, and we use Conv-LSTM to integrate the output of all Temp-LSTMs to obtain multi-level data encoded in hidden layers. Experiments were conducted on two benchmark databases, Oulu-CASIA, and, SASE-FE, however the results showed that the proposed method achieved better performance than the expanded versions of CNN, 3D CNN, and Recurrent methods.

Keyphrases: 3D CNN, LSTM, deep learning, emotions recognition, facial expressions

@booklet{EasyChair:9929,
  author    = {Hmad Zennou and Mohamed Ouhda and Mohamed Baslam},
  title     = {Toward an Efficient Emotion Recognition from Facial Expressions Using ML},
  howpublished = {EasyChair Preprint 9929},
  year      = {EasyChair, 2023}}