In this paper, a novel tool for facial and gesture analysis aiming to quantify different
subjective measures employed in the Speech Language Pathologist area is proposed.
Through an input video (from a simple monocular camera) showing a person's face, the
developed tool can track the movements and expressions from it in order to extract
useful morphological and gestural parameters that are of interest in different fields
of study, such as, Speech Language Pathologist, Neurology, etc. A modified version
of a 3D face model, named Candide-3, is employed in the tracking stage. Since the
original 3D model cannot handle asymmetrical facial movements, a new set of animation
units was implemented in order to effectively track asymmetrical gestures. To enhance
the tracking accuracy, a fusion scheme is proposed in the facial gesture tracking
stage by means of the combination of the 3D face model previously described and facial
landmarks detected using deep learning models. This tool will be made open source,
both the software application (oriented to health professionals, no need to have any
programming knowledge), and the source code for the computer vision community. Several
perceptual experiments were carried out, achieving promising results.

Micro-expressions describe unconscious facial movements which reflect a person's psychological
state even when there is an attempt to conceal it. Often used in psychological and
forensic applications, their manual recognition requires professional training and
is time consuming. Therefore, achieving automatic recognition by means of computer
vision would confer enormous benefits. Facial Action Unit (AU) is a coding of facial
muscular complexes which can be independently activated. Each AU represents a specific
facial action. In the present paper, we propose a method for the challenging task
that is the detection of activated AUs when the micro-expression occurs, which is
crucial in the inference of emotion from a video capturing a micro-expression. This
specific problem is made all the more difficult in the light of limited amounts of
data available and the subtlety of micro-movements. We propose a segmentation method
for key facial sub-regions based on the location of AUs and facial landmarks, which
extracts 11 facial key regions from each sequence of micro-expression images. AUs
are assigned to different local areas for multi-label classification. Considering
that there is little prior work on the specific task of detection of AU activation
in the existing literature on micro-expression analysis, for the evaluation of the
proposed method we design an AU independent cross-validation method and adopt Unweighted
Average Recall (UAR), Unweighted F1-score (UF1), and their average as the scoring
criteria. Evaluated using the established standards in the field and compared with
previous work, our approach is shown to exhibit state-of-the-art performance.

Facial micro-expression computational analyses are becoming a prevalent research area,
automatically micro-expression spotting as the first-in-the-pipeline problem has not
been resolved yet. There are two main factors that confine the performance of current
studies. 1) Subtle involuntary movements of micro-expression are hard to capture.
2) Micro-expression datasets are relatively small that can not fully support the training
of deep neural networks. For the first problem, we propose modeling the expression
movements from the view of consecutive frames in the wavelet space as temporal features.
Combined with spatial features encoded by a convolutional neural network, temporal
and spatial information can supplement each other in further analyses. For the second
problem, we adopt transfer learning from other emotion-related tasks since the facial
prior is homologous to our task. To train our model, we covert the spotting task to
a frame-level classification task, meanwhile, weighted focal loss is used to deal
with severe class imbalance. With leave-one-subject-out cross-validation, our method
reports F1-score of 0.1763 and 0.1360 for CAS(ME)2 and SAMM-LV respectively. Code
is available at https://github.com/guanjz20/MM21_FME_solution.

Emotional detection based on facial expressions is an important procedure in high-risk
tasks such as criminal investigation or lie detection. To reduce the impact of the
inconsistency in the duration of macro- and micro-expression, we propose an effective
Spatio-temporal Convolutional Attention Network (STCAN) for spotting macro- and micro-expression
intervals in long video sequences. The spatial features of each image in the video
sequence are extracted through the Convolution Neural Network. Then, considering the
problem of the inconsistency in the duration of the macro- and micro-expression, the
multi-head self-attention model is used to analyze the weight of the spatial feature
of the image in the temporal space. Finally, the time interval of emotional changes
is determined according to the weight of each frame of the video sequence, and the
macro- and micro-expression intervals are obtained through the threshold segmentation
model. Considering the problem of Leave-One-Subject-Out cross-validation the training
time long, we verified the effectiveness of our model on the SAMM Long Video and CAS(ME)2
datasets through the Leave-Half-Subject-Out (LHSO) cross-validation method. The experiments
show that the STCAN model can achieve competitive results on Facial Micro-Expression
(FME) Challenge 2021.

Facial expressions are an important way for humans to perceive emotions. The advent
of facial action coding systems has enabled the quantification of facial expressions.
Moreover, a large amount of annotated data facilitates the performance of deep learning
for the spotting and recognition of expressions or micro-expressions. However, the
study of video-based expressions or micro-expressions requires coders to have expertise
while also familiar with action unit (AU) coding. This paper systematically sorts
out the relationship between facial muscles and AU to make more people understand
AU coding from the principle. For this purpose, we have made a brief guide to get
started as quickly as possible for the beginner to code.