Chinese sign language recognition based on surface electromyography and motion information

• Published in: PloS one Vol. 18; no. 12; p. e0295398
• Main Authors: Li, Wenyu, Luo, Zhizeng, Li, Wenguo, Xi, Xugang
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 07-12-2023; Public Library of Science (PLoS)
• Subjects: Algorithms; Analysis; China; Data integration; Electromyography; Evaluation; Gestures; Hand; Humans; Language; Markov chains; Markov processes; Pattern Recognition, Automated; Recognition; Sensors; Sign Language; Trajectories; Voice recognition; China
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0295398

Sign language (SL) has strong structural features. Various gestures and the complex trajectories of hand movements bring challenges to sign language recognition (SLR). Based on the inherent correlation between gesture and trajectory of SL action, SLR is organically divided into gesture-based recognition and gesture-related movement trajectory recognition. One hundred and twenty commonly used Chinese SL words involving 9 gestures and 8 movement trajectories, are selected as research and test objects. The method based on the amplitude state of surface electromyography (sEMG) signal and acceleration signal is used for vocabulary segmentation. The multi-sensor decision fusion method of coupled hidden Markov model is used to complete the recognition of SL vocabulary, and the average recognition rate is 90.41%. Experiments show that the method of sEMG signal and motion information fusion has good practicability in SLR.