Title: A scale space model of weighted average CNN ensemble for ASL fingerspelling recognition

Authors: Neena Aloysius; M. Geetha

Addresses: Department of Computer Science and Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India ' Department of Computer Science and Engineering, Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India

Abstract: A sign language recognition system facilitates communication between the deaf community and the hearing majority. This paper proposes a novel specialised convolutional neural network (CNN) model, SignNet, to recognise hand gesture signs by incorporating scale space theory to deep learning framework. The proposed model is a weighted average ensemble of CNNs – a low resolution network (LRN), an intermediate resolution network (IRN) and a high resolution network (HRN). Augmented versions of VGG-16 are used as LRN, IRN and HRN. The ensemble works at different spatial resolutions and at varying depths of CNN. The SignNet model was assessed with static signs of American Sign Language – alphabets and digits. Since there exists no sign dataset for deep learning, the ensemble performance is evaluated on the synthetic dataset which we have collected for this task. Assessment of the synthetic dataset by SignNet reported an impressive accuracy of over 92%, notably superior to the other existing models.

Keywords: convolutional neural networks; CNNs; sign language; fingerspelling; ensemble; vgg-16; classification; scale space; spatial resolution.

DOI: 10.1504/IJCSE.2020.107268

International Journal of Computational Science and Engineering, 2020 Vol.22 No.1, pp.154 - 161

Received: 02 Apr 2019
Accepted: 03 Jul 2019
Published online: 11 May 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article