okonp07/South-African-Languages-Hackathon

A model that Identifies the language in which a given text is written # EXPLORE Data Science Academy Classification Hackathon ## Project Overview South Africa is a multicultural society that is characterised by its rich linguistic diversity. Language is an indispensable tool that can be used to deepen democracy and also contribute to the social, cultural, intellectual, economic and political life of the South African society. The country is multilingual with 11 official languages, each of which is guaranteed equal status. Most South Africans are multilingual and able to speak at least two or more of the official languages. From South African Government ## Student Name / Supervisor **Student: Okon Prince** **Supervisor: John Mohale** ## Project Aim To build a model that will accurately classify any text fed into it which is written in any of South Africas 11 official languages into the proper language category that it belongs to ## Data Details The dataset used for this challenge is the NCHLT Text Corpora collected by the South African Department of Arts and Culture & Centre for Text Technology (CTexT, North-West University, South Africa). The training set was improved through additional cleaning done by Praekelt. The data is in the form Language ID, Text. The text is in various states of cleanliness. Some NLP techniques will be necessary to clean up the data. ### Data Source The collection of this data was funded by the Canada Foundation for Innovation JELF Grant to Chris Bauch, University of Waterloo. The dataset aggregates tweets pertaining to climate change collected between Apr 27, 2015 and Feb 21, 2018. In total, 43943 tweets were collected. Each tweet is labelled as one of the following classes: ### Class Description The classes are 11 distinct languages, and the Language IDs are: *** afr - Afrikaans eng - English nbl - isiNdebele nso - Sepedi sot - Sesotho ssw - siSwati tsn - Setswana tso - Xitsonga ven - Tshivenda xho - isiXhosa zul - isiZulu *** ### Jupyter Note book File (Project Base file) The jupyter notebook i …