Title: Multi-stream fusion network for continuous gesture recognition based on sEMG

Authors: Jun Li; Chunlong Zou; Dalai Tang; Ying Sun; Hanwen Fan; Boao Li; Xinjie Tang

Addresses: Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, Hubei, China ' College of Mechanical Engineering, Hubei University of Automotive Technology, Shiyan, Hubei, China ' Inner Mongolia University of Finance and Economics, Ancient Hohhot, China ' Research Centre for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, Hubei, China ' Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, Hubei, China ' Research Centre for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, Hubei, China ' Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, Hubei, China

Abstract: Dynamic gesture recognition is an essential step in human-computer interaction. However, dynamic gesture recognition based on Surface Electromyography (sEMG) signals faces issues such as incomplete feature extraction and incomplete recognition of temporal information between continuous gestures. Therefore, this paper proposes a multi-stream fusion network (MSK-LCNN) for dynamic gesture recognition to improve accuracy. We combine CNN and LSTM models into a unified framework which extracts spatio-temporal information both globally and in depth, and combines feature fusion to retain essential information. This framework uses an attention mechanism (SKNet) to learn more intricate feature information. The gesture recognition accuracy of this method on our data set is 95.23%, and on the NinaPro DB1 data set, it is 91.45%, which outperforms other similar networks in recent years. Applying this algorithm to prosthetic hand control has achieved flexible and stable control of the prosthetic hand.

Keywords: sEMG; gesture recognition; attention mechanisms; neural networks; multi-stream network.

DOI: 10.1504/IJWMC.2024.138853

International Journal of Wireless and Mobile Computing, 2024 Vol.26 No.4, pp.374 - 383

Received: 17 Nov 2023
Accepted: 04 Jan 2024
Published online: 31 May 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article