Title: Virtual guitar: using real-time finger tracking for musical instruments

Authors: Noorkholis Luthfil Hakim; Shih-Wei Sun; Mu-Hsen Hsu; Timothy K. Shih; Shih-Jung Wu

Addresses: Department of Computer Science and Information Engineering, National Central University, Zhongli City, Taoyuan, Taiwan ' Department of New Media Art, Taipei National University of the Arts, Taipei, Taiwan ' Department of Computer Science and Information Engineering, National Central University, Zhongli City, Taoyuan, Taiwan ' Department of Computer Science and Information Engineering, National Central University, Zhongli City, Taoyuan, Taiwan ' Department of Innovative Information and Technology, Tamkang University (TKU), Lanyang Campus, Taipei, Taiwan

Abstract: Kinect, a 3D sensing device from Microsoft, invokes the human-computer interaction research evolution. Kinect was implemented in many areas including music. One was implemented in a virtual musical instrument (VMI) system, which uses natural gestures to produce synthetic sounds similar to real musical instruments. From related work, it was found that the use of a large joint such as hand, arm and leg is inconvenient and limits the way of playing VMI. Thus, this study proposes a fast and reliable finger tracking algorithm suitable for virtual musical instrument playing. In addition, a virtual guitar system application is developed as an implementation of the proposed algorithm. Experimental results show that the proposed method can be used to play many varieties of songs with an acceptable quality. Furthermore proposed application could be used by a beginner who does not have any experience in music or on playing a real musical instrument.

Keywords: virtual guitar; finger tracking; human computer interaction; hand detection; hand tracking; hand gesture recognition; virtual musical instrument; VMI; depth camera.

DOI: 10.1504/IJCSE.2019.099081

International Journal of Computational Science and Engineering, 2019 Vol.18 No.4, pp.438 - 450

Received: 26 May 2016
Accepted: 11 Oct 2016
Published online: 15 Apr 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article