Toluwase/Word-Level-Language-Identification-for-Resource-Scarce-

English, Hausa, Igbo and Yoruba corpora and results (presented in excel files) of word-level language identification research using the character trigram of the featured languages # Word-Level Language Identification for Resource-Scarce Languages Definition of Terms 'This repository' refers to all the resources which are accessed through this hyperlink or url: github.com The word-level language identification in this study refers to identifying the language of words in texts. This study uses the term the main language, which refers to the language of interest. The scope of this study is two languages; one language of interest and one foreign language. In essence, a text will be in two languages: the main language and foreign language. For instance, if the main language is Yoruba, foreign language could be English. In this case text will be in Yoruba while there will be some words in English. Introduction This README is a description of the datasets, research results and python program contained on this page; the datasets were used for the word-level language identification research for resource-scarce languages. The word-level language identification strategy proposed in the research was performed through the pattern analysis of the character trigrams of the featured languages. Languages featured in this research are English, Hausa, Igbo and Yoruba. The word-level language identification strategy does not require large corpus which are needed in/for previous word-level language identification and other natural language processing research. Secondly, this strategy potentially works for all languages whose writing systems are alphabet-based. File Manifest There are three types of Files Python Program: The Python program is contained in a python file named trigramanalyzer_wordlanguageid.py. Kindly refer to the Operating Instructions for details of the Python program Corpora: One of the contributions of this research is the corpora of the three Nigerian languages (Hausa, Igbo and Yoruba) which were used for this research and we are making available to the public through t …