Title: Real time sign language recognition using depth sensor

Authors: Jayesh Gangrade; Jyoti Bharti

Addresses: Department of Computer Science and Engineering, Maunala Azad National Institute of Technology, Bhopal, India; IES IPS Academy, Indore, 452012, India ' Department of Computer Science and Engineering, Maunala Azad National Institute of Technology, Bhopal, India

Abstract: Communication via gestures is a visual dialect utilised by deaf and hard-of-hearing (HoH) people group. This paper proposed a system for sign language recognition utilising human skeleton data provided from Microsoft's Kinect sensor to recognising sign gestures. The Kinect sensor generates the skeleton of a human body and distinguishes 20 joints in it. The proposed method utilises 11 out of 20 joints and extracts 35 novel features per frame, based on distances, angles and velocity involving upper body joints. Multi-class support vector machine classified the 35 Indian sign gestures in real time with accuracy of 87.6%. The proposed method is robust in cluttered environment and viewpoint variation.

Keywords: Indian sign gesture; multi-class support vector machine; human computer interaction; pattern recognition; Kinect sensor.

DOI: 10.1504/IJCVR.2019.101527

International Journal of Computational Vision and Robotics, 2019 Vol.9 No.4, pp.329 - 339

Received: 23 Sep 2017
Accepted: 03 May 2018
Published online: 12 Aug 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article