rmulumba/malawi_text_classification

Classifying text in Chichewa language from Malawi. # AI4D Malawi News Classification Challenge ## Classifying news articles in Chichewa This project is a response to Zindi's AI4D Malawi News Classification Challenge. More information about this challenge can be found here. ## Data source The data used in this project is from the Zindi AI4D Malawi News Classification Challenge. ## Requirements 1. Pandas - `pip install pandas` 2. Numpy - `pip install numpy` 3. Scikit-learn - `pip install scikit-learn` 4. Torch - `pip install torch` 5. Transformers - `pip install transformers` 6. Sentencepiece - `pip install sentencepiece` 7. nltk - `pip install nltk` ## Approach I used a pretrained xlnet-base-cased model train a classification algorithm. ## Execution Run `train_logistic_reg.py` in the `src` directory. ## References: 1. Approaching (Almost) Any Machine Learning Problem - Abhishek Thakur 2. Top 6 Open Source Pretrained Models for Text Classification you should use 3. Fine-tuning XLNet language model to get better results on text classification 4. High accuracy text classification with Python 5. Sentiment Analysis (Opinion Mining) with Python — NLP Tutorial