NEMLAR CORPUS IMPROVEMENT FOR ARABIC NATURAL LANGUAGE PROCESSING

Most machine learning approaches in Natural Language Processing rely mainly on corpora. Indeed, various applications based on this approaches require prior learning of statistical models, including the Hidden Markov Model for Part Of Speech Tagging. However, this learning resources must meet some criteria to have a well trained model, and thus more accurate results. On the other hand, we find that the Arabic language - despite its vast use on the internet and in social media - has a limited number of linguistic resources for machine learning, especially corpora with morpho-syntactic annotations. Thus, in this article we will treat the Nemlar corpus, one of the richest annotated linguistic corpora for the Arabic language. The aimed version will have several contributions, especially increasing the rate of recognized words and, subsequently, reducing Out Of Vocabulary words (which represents a major problems in many NLP tasks); as well as fine-grain tagging, by separating the words into their smallest possible sub-units, which will open the way to new applications relying on the granular aspect of Arabic. In this article, we will first present the content of the Nemlar corpus. We will then define some criteria in order to improve its structure and enrich its content. We will also present the different modifications made on the original version, including merging POS tags, separating prefixes and suffixes, creating tags for specific cases, etc. in order to lead to the desired form. Then, we will see the experimentation evaluating the new word recognition rate. At the end, we will talk about the advantages and disadvantages of the resulting version.