Title: Sign language recognition in complex background scene based on adaptive skin colour modelling and support vector machine

Authors: Tse-Yu Pan; Li-Yun Lo; Chung-Wei Yeh; Jhe-Wei Li; Hou-Tim Liu; Min-Chun Hu

Addresses: Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan ' Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan ' Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan ' Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan ' Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan ' Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan

Abstract: With the advances of wearable cameras, the user can record the first-person view videos for gesture recognition or even sign language recognition to help the deaf or hard of hearing people communicate with others. In this paper, we propose a purely vision-based sign language recognition system which can be used in complex background scene. We design an adaptive skin colour modelling method for hand segmentation so that the hand contour can be derived more accurately even when different users use our system in various light conditions. Four kinds of feature descriptors are integrated to describe the contours and the salient points of hand gestures, and support vector machine (SVM) is applied to classify hand gestures. Our recognition method is evaluated by two datasets: 1) the CSL dataset collected by ourselves in which images were captured in three different environments including complex background; 2) the public ASL dataset, in which images of the same gesture were captured in different lighting conditions. The proposed recognition method achieves acceptable accuracy rates of 100.0% and 94.0% for the CSL and ASL datasets, respectively.

Keywords: sign language recognition; support vector machine; SVM; human-computer interaction; gesture recognition.

DOI: 10.1504/IJBDI.2018.088277

International Journal of Big Data Intelligence, 2018 Vol.5 No.1/2, pp.21 - 30

Received: 28 Apr 2016
Accepted: 17 Oct 2016
Published online: 01 Dec 2017 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article