Title: Facial micro-expression recognition method based on CNN and transformer mixed model

Authors: Yi Tang; Jiaojun Yi; Feigang Tan

Addresses: School of Information Technology and Engineering, Guangzhou College of Commerce, Guangzhou, 511363, China ' School of Economics, Guangzhou College of Commerce, Guangzhou, 511363, China ' School of Traffic and Environment, Shenzhen Institute of Information Technology, Shenzhen, 518172, China

Abstract: The existing methods for facial microexpression recognition have the problem of low efficiency and accuracy. Therefore, a facial micro-expression recognition method based on a hybrid model of CNN and transformer is proposed. Extract facial hierarchical features using a hybrid model of CNN and transformer, and use them as inputs to a deep network. At the same time, the facial microexpression image area is segmented and the image is smoothed through threshold to obtain the feature vectors of the facial microexpression. These feature vectors are input into a CNN and transformer hybrid model to achieve recognition of facial microexpressions. The experimental results show that the proposed method can recognise facial microexpressions in complete or incomplete images, and the recognition state delay is controlled below 5 ms. In addition, compared to traditional methods, this method has a higher average recognition accuracy, up to 98%.

Keywords: CNN; transformer mixed model; micro-expression of human face; recognition method.

DOI: 10.1504/IJBM.2024.140771

International Journal of Biometrics, 2024 Vol.16 No.5, pp.463 - 477

Received: 06 Jul 2023
Accepted: 14 Sep 2023
Published online: 02 Sep 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article