Published on Mon Jul 20 2020

Landmark Guidance Independent Spatio-channel Attention and Complementary Context Information based Facial Expression Recognition

Darshan Gera, S Balasubramanian

Modern facial expression recognition(FER) architectures rely on external sources like landmark detectors for defining attention. The proposed architecture obtains both local and global attention per channel per spatial location through a novel spatio-channel attention net. The representation learnt by the proposed architecture is robust to occlusions and pose variations.

0
0
0
Abstract

A recent trend to recognize facial expressions in the real-world scenario is to deploy attention based convolutional neural networks (CNNs) locally to signify the importance of facial regions and, combine it with global facial features and/or other complementary context information for performance gain. However, in the presence of occlusions and pose variations, different channels respond differently, and further that the response intensity of a channel differ across spatial locations. Also, modern facial expression recognition(FER) architectures rely on external sources like landmark detectors for defining attention. Failure of landmark detector will have a cascading effect on FER. Additionally, there is no emphasis laid on the relevance of features that are input to compute complementary context information. Leveraging on the aforementioned observations, an end-to-end architecture for FER is proposed in this work that obtains both local and global attention per channel per spatial location through a novel spatio-channel attention net (SCAN), without seeking any information from the landmark detectors. SCAN is complemented by a complementary context information (CCI) branch. Further, using efficient channel attention (ECA), the relevance of features input to CCI is also attended to. The representation learnt by the proposed architecture is robust to occlusions and pose variations. Robustness and superior performance of the proposed model is demonstrated on both in-lab and in-the-wild datasets (AffectNet, FERPlus, RAF-DB, FED-RO, SFEW, CK+, Oulu-CASIA and JAFFE) along with a couple of constructed face mask datasets resembling masked faces in COVID-19 scenario. Codes are publicly available at https://github.com/1980x/SCAN-CCI-FER

Tue Sep 29 2020
Computer Vision
Affect Expression Behaviour Analysis in the Wild using Spatio-Channel Attention and Complementary Context Information
Facial expression recognition(FER) in the wild is crucial for building reliable human-computer interactive systems. Current FER systems fail to perform well under various natural and un-controlled conditions. Spatial-channel attention net(SCAN) is used to extract local and global attentive features.
0
0
0
Wed Mar 31 2021
Computer Vision
Robust Facial Expression Recognition with Convolutional Visual Transformers
Facial Expression Recognition (FER) in the wild is extremely challenging due to occlusions, variant head poses, face deformation and motion blur. We propose Convolutional Visual Transformers to tackle FER by two main steps. First, we propose an attentional selective fusion (ASF) for leveraging the feature maps generated by two-branch CNNs.
3
1
1
Tue Jun 08 2021
Computer Vision
MViT: Mask Vision Transformer for Facial Expression Recognition in the wild
Facial Expression Recognition (FER) in the wild is an extremely challenging task in computer vision. The self-attention mechanism makes transformers obtain a global receptive field in the first layer. This dramatically enhances the feature extraction capability of transformers.
5
0
0
Fri May 10 2019
Computer Vision
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Occlusion and pose variations are two major obstacles for automatic Facial Expression Recognition (FER) This paper addresses the real-world pose and occlusion robust FER problem with three-fold contributions. We build several in-the-wild facial expression datasets with manual annotations for the community. Second, we propose a novel Region Attention Network (
0
0
0
Fri Jan 31 2020
Computer Vision
Lossless Attention in Convolutional Networks for Facial Expression Recognition in the Wild
Facial expressions recognition in the wild is a challenging task and existing methods can't perform well. We propose a Lossless Attention Model for convolutional neural networks to extract attention-aware features from faces.
0
0
0
Mon Dec 07 2020
Computer Vision
MERANet: Facial Micro-Expression Recognition using 3D Residual Attention Network
We propose a facial micro-expression recognition model using 3D residualattention network called MERANet. The proposed model takes advantage of geographically-temporal attention and channel attention together, to learn deeper-grained subtle features for classification of emotions.
0
0
0