Understanding a language and how its native speakers speak it requires more than just knowing the words and their meanings. It involves understanding the context, nuances, and cultural aspects that give the language its depth. When we want a machine to “speak” or “understand” a language, it needs to appreciate the elements of phonology, morphology, and semantics, as well as the cultural aspects of the language. Language is an entity that breathes life into everyday human interaction.
Let’s take the phrase, “Come put it on my head”. To an outsider or a basic NLP system, this might sound puzzling or even nonsensical. But in Nigeria, many people know what it means, especially when used by an angry mother. To them, it’s a clear expression of frustration or disbelief. This seemingly innocent phrase perfectly illustrates the complexities of languages, as it showcases how cultural context can completely transform the meaning of words.
In order to accurately capture language for NLP systems at the level of a native speaker, it is crucial to involve linguists. Linguists play a vital role in providing context by understanding that language involves more than just its words. It also involves context, emotion, and cultural significance.
Linguistic Challenges Presented by African Languages
Morphological Complexity
African languages are fascinating because they often use a lot of different ways to change and build words, making them rich and complex. This happens through aspects like adding prefixes, suffixes, or even repeating parts of words to bring out different meanings or grammatical details and nuances. For instance, in Kiswahili, a single verb can be transformed to reflect not only the tense but also the subject, object, mood, and even the relationship between the speaker and listener. For example, the verb “kula” (which means “to eat”) can change in many ways depending on who is eating, when they’re eating, and how the action relates to others. “Nilikula” means “I ate,” “atakula” means “he or she will eat,” and “tunakula” means “we are eating.”
Each change involves a series of set rules that add layers of meaning, making it challenging for NLP models to capture all the possible variations.
Multilingualism
It is common to hear someone using two or more languages in a single conversation. This may occur when the initial language used does not effectively convey the intended message. For instance, an individual may initiate a conversation in Kiswahili, switch to their mother tongue, and conclude in English. This is quite normal, especially in urban areas where various cultures and languages coexist and regularly interact. Sometimes, this feeds into the evolution of pidgin, sheng, or creole languages specific to the sub-regions within the local cities and towns.
Code-switching, as linguists call it, is more than just a habit—it’s a way of expressing identity, navigating social situations, and conveying nuanced meanings that might be lost in a single language.
Tonal Variations
In many African languages, the tone or pitch of a word can completely change its meaning. For example, in Yoruba, the word “odo” can mean zero or river, depending on which ‘o’ you stress and stretch. Getting an NLP system to understand the difference between these two words and translate them effectively would require it to appreciate nuances, which can be quite difficult for them to grasp.
Oral Traditions
African communities have always depended on oral traditions to pass down their histories and culture to future generations. It’s a way of life. Your grandma tells you stories in your mother tongue that stick with you, but you can’t later reference them as they are not captured in any books or texts. This makes it tough for NLP systems to get the training data they need to learn some of these languages. There is an opportunity for field linguists to interact with such communities to document languages that have very few existing datasets. Field linguistics has helped digitize languages across the world that were once considered inaccessible, with technology providing prosperity for these languages.
In addition to the challenges mentioned earlier, the presence of dialects, idioms, metaphors, and borrowed words further complicates the task for NLP systems. Therefore, the involvement of linguists and local communities is crucial for tasks such as collecting and transcribing oral traditions, updating language datasets with new usage, and outlining language rules and patterns. This would enable NLP tools to better comprehend and process the various African languages.
Advancements in NLP tools due to collaboration with Linguists
Growth in Speech Recognition Tools: There has been significant progress in speech recognition tools, thanks to the collaboration between linguists and tech developers. This collaboration has particularly benefited African languages, as it has enabled natural language processing (NLP) tools to better understand and interpret the nuances of pronunciation, accents, and tones in these languages. As a result, these systems have become more accurate in recognizing and interpreting the speech patterns and accents of African users.
Companies like Ambani Africa are leveraging such technology to educate children in their native languages, thereby creating a bridge between traditional education and modern technology. Intron Health is using speech-to-text for medical documentation, supporting more than 200 African accents.
Improvement of Translation Services: There was a time when translation tools often produced comically inaccurate results when translating to or from African languages. However, by working with linguists, these models have improved in capturing the intended meaning and cultural context of the text. This collaboration has significantly enhanced translation tools, making them more practical for everyday communication in African languages. There is still a long way to go, but progress is being made.
Digital Preservation of Endangered Languages – There is something incredibly precious about a language that has been spoken for centuries and passed down from generation to generation. However, many African languages are disappearing, taking with them unique histories, traditions, and perspectives. For example, the Khoekhoe language, with around 200,000 speakers, is quickly becoming endangered. It is being preserved in digital form in the Hugh Brody collection, along with a few other languages. This preservation gives it a chance to exist in the digital space and offers hope for its revitalization on a massive scale through technology.
To create truly inclusive and effective NLP tools, it is important to embrace and appreciate the unique personality of African languages. By fostering collaborations between linguists and language communities, who have a deep understanding of the essence of the language, and NLP technologists who can bring it to life in the digital space, we can provide solutions to billions of users on the continent. This will enable them to fully participate in the digital world with solutions that reflect their way of life.
About Lanfrica
Lanfrica is a platform that aggregates resources on African languages, making them easily discoverable and accessible. We believe that by connecting and linking these resources, we can help researchers avoid unnecessary duplication of efforts and thus foster greater collaboration. This, in turn, will lead to more development of language technologies that respect and leverage the unique characteristics of African languages. Follow us on X(formerly Twitter), LinkedIn or join the community Slack channel to become part of this journey!