End-to-End Continuous Ethiopia Sign Language Recognition

This paper tackles the challenge of continuous Ethiopian Sign Language (EthSL) recognition, a vital yet complex form of communication for Deaf communities, due to its dynamic gestures and the variability between signers. While previous research has primarily focused on isolated signs at the character, word, or phrase levels, continuous sentence-level recognition remains underexplored. To address this gap, we introduce the first dataset specifically designed for continuous, multi-signer EthSL, consisting of 1,320 video sequences representing 30 unique sentences, signed by 22 individuals, each sentence performed twice by each signer. Our proposed framework utilizes an end-to-end architecture that begins with 2D convolutional layers to extract spatial features from video frames, followed by 1D convolutional layers to capture short-term temporal patterns. These features are processed by a Bidirectional Long Short-Term Memory (BiLSTM) network to capture long-term temporal dependencies necessary for continuous EthSL recognition. The final classifier employs Connectionist Temporal Classification (CTC) to align and decode the sign sequence into coherent, sentence-level outputs. The proposed model achieved a Word Error Rate (WER) of 8.82% in a signer-independent setup and 47.02% on an unseen sentence split, demonstrating strong performance across signer variations, though with room for improvement in generalizing to novel sentences. These findings highlight the potential of our approach for practical applications, such as automated translation and assistive communication tools for the Deaf community. Future research will focus on enhancing recognition accuracy by expanding the dataset with a broader range of sentence structures and signing styles, aiming to create more adaptable EthSL recognition systems. Our dataset and source codes are available at here