REALTIME ETHIOPIAN SIGN LANGUAGE ALPHABET RECOGNITION VIA COMPUTER VISION AND MOTION SENSOR

Unlike ASL or other Sign language alphabets, ESL fingerspelling is primely distinguished by hand shape and hand motion. ESL finger spelling represents consonant series with hand configurations, seven movements correspond to the seven vowels. This makes Ethiopian sign language dynamic. In dynamic sign languages, motion detection is essential for the recognition of the language. In this work, an ESL fingerspelling recognition using a fusing of computer vision and motion sensor research work is proposed to recognize and classify the vowel using motion sensor data and deep learning, also using computer vision to recognize consonants. With A large dataset of Ethiopia manual sign language alphabets from multiple subjects, two deep neural networks selected to derive a best topology for each. The vison model is implemented by a YoloV5 model and the smartwatch motion sensor motion model is implemented by LSTM network