JasperZeroes/language_identification_hackathon

This project involves building a classification model to identify the various South African languages # NLP's Language Identification Hackathon This is a Natural Language Processing, NLP, classification project under Supervised Machine Learning which involves building a model to determine the natural language that a piece of text is written in. In this context, we'll be determining which of South Africa's 11 Official languages a particular text is written in. ## Project Introduction --- South Africa is a multicultural society that is characterised by its rich linguistic diversity. Language is an indispensable tool that can be used to deepen democracy and also contribute to the social, cultural, intellectual, economic and political life of the South African society. The country is multilingual with 11 official languages, each of which is guaranteed equal status. Most South Africans are multilingual and able to speak at least two or more of the official languages. With such a multilingual population, it is only obvious that our systems and devices also communicate in multi-languages. In this project, we took text which is in any of South Africa's 11 Official languages and identify which language the text is in. ### Where is this data from? The dataset used for this project is the NCHLT Text Corpora collected by the South African Department of Arts and Culture & Centre for Text Technology (CTexT, North-West University, South Africa). The training set was improved through additional cleaning done by Praekelt. The text is in various states of cleanliness. Some NLP techniques will be necessary to clean up the data. ### Class Description Below are the 11 languages contained in the train_set.csv file 1. afr - Afrikaans 2. eng - English 3. nbl - isiNdebele 4. nso - Sepedi 5. sot - Sesotho 6. ssw - siSwati 7. tsn - Setswana 8. tso - Xitsonga 9. ven - Tshivenda 10. xho - isiXhosa 11. zul - isiZulu ### Features * language id: A three-letter word to denote a language * text: message body ---