Tanzanian sign language recognition system for an assistive communication glove sign tutor based on the inertial sensor fusion control algorithm

Abstract This paper presents a sign language recognition system for sign tutoring assistive hand data gloves for hearing-impaired people. In this study, specially designed 5-fingered data gloves are used for interaction and communication with hearing-impaired or hard-of-hearing people using signs. In this paper, a sign language recognition scheme based on an inertial sensor fusion control algorithm is proposed to achieve efficient and accurate real-time sign language recognition. The fusion control algorithm uses a feedback control idea to fuse two traditional attitude information calculation methods, reducing the impact of the environment on the sensor. The attitude information of the tested object in the instantaneous state can be accurately obtained. The algorithm uses the classification methods of support vector machine (SVM), K-nearest neighbor method (KNN) and feedforward neural network (FNN) classifier adaptive model to classify the data collected by the sign language data through data fusion, data preprocessing and feature extraction. The results show that the proposed sensor fusion control algorithm effectively obtains real-time poses. The recognition accuracy of the sign language recognition scheme for 26 kinds of Tanzanian sign languages is 96.5%. The proposed scheme will lay a solid foundation for sign language recognition systems and provide a reference for relevant research on sensor fusion control.