Giving a voice: A novel ecosystem approach combining visual and sensor based applications to sign language translation

International audience Verbal communication serves as the main form of communication for most people around the world, however, people from deaf and mute communities find difficulty using this medium of communication and consequently find it difficult to communicate with the majority of people. Furthermore, people also find difficulty with finding a suitable translator that can efficiently and seamlessly translate sign language to spoken words. This research aims to propose a new design for a wearable sign language translator utilizing a wearable neck device which contains speakers, integrated camera, a LiDAR sensor in combination with a small accelerometer placed in the forearms, all of which track and analyze the user’s hand and finger movement to translate the user’s sign language to the corresponding text. Through the use of text-to-speech synthesis, this text is then played out through the speakers to nearby listeners, effectively bridging the gap between sign language users and the rest of the population.