Title: A hybrid deep learning approach for online learners' recognition using screen name, and learners' emotion detection

Authors: Purushottama Rao K.; B. Janet

Addresses: Department of Computer Applications, National Institute of Technology Tiruchirappalli, Tamilnadu – 620015, India ' Department of Computer Applications, National Institute of Technology Tiruchirappalli, Tamilnadu – 620015, India

Abstract: Online learning environments (OLEs) have enabled global access to education during and after the COVID-19 pandemic. While widely accepted, OLEs pose challenges for teachers in recognising students and understanding their emotions, unlike in traditional classrooms. To address this, a hybrid deep learning architecture is proposed that uses screen names and facial expressions to identify students and detect their emotional states. The MTCNN model is used to segment the frames of an online class video where each segmented part contains a student's face and screen name. The EAST and Py-Tesseract and face emotion recognition models are then applied on these segmented parts to extract screen names and detect the emotional states of the students. This helps in detecting individual student's emotional feedback and overall class feedback. Finally, the students are classified as engaged or not engaged. The proposed architecture will make OLEs as efficient as traditional classrooms.

Keywords: deep learning; face detection; facial emotion detection; online learning environments; OLEs; person identification; text extraction.

DOI: 10.1504/IJTEL.2024.139687

International Journal of Technology Enhanced Learning, 2024 Vol.16 No.3, pp.235 - 252

Received: 16 Jun 2023
Accepted: 16 Jul 2023
Published online: 05 Jul 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article