labrijisaad/Sentiment-Analysis-model-for-the-Wolof-language

In this notebook, I tried to create a sentiment analysis model for Wolof language. # 📈 `Sentiment Analysis model for the Wolof language` 🌍: - 🎯 In this notebook, I tried to create a **`sentiment analysis model for Wolof language`**. - 🤔 To do this, I gathered data already labeled in different datasets extracted from different sources (Quran, Bible, proverbs, twitter): The notebooks I used for web scraping can be found in my github repository). - ⏭ Next, i used some of scikit learn's most well-known classification models with **`TF-IDF Vectorizer`** to train the final model: - **`Logistic Regression`** - **`Bernoulli Naive Bayes`** - **`Decision Tree`** - **`Support vector machine`** - Here are **TWO** ways to use the trained model in notebook: (You must before install the requirements) ##### meth 1 > via model and vectorizer import ```py import pickle import pandas as pd import re SVM_model = pickle.load(open('SVM_model.pkl', 'rb')) SVM_vectorizer = pickle.load(open("SVM_vectorizer.pk","rb")) LR_model = pickle.load(open('LR_model.pkl', 'rb')) LR_vectorizer = pickle.load(open("LR_vectorizer.pk","rb")) def process_wolof_text(text): ## text processing text = text.lower() text = remove_ponctuation(text) text = cleaning_numbers(text) text = cleaning_stopwords(text) text = remove_ponctuation(text) text = re.sub("\s\s+", " ", text) return text def predict_sentiment_svm(text): serie = pd.Series(text) vector = SVM_vectorizer.transform(serie) return str(SVM_model.predict(vector)[0]) def predict_wolof_with_logistic_regression(text): text = process_wolof_text(text) text = [text] text = LR_vectorizer.transform(text) return "POSITIVE" if (LR_model.predict(text)[0] == 1) else "NEGATIVE" text = "dafa xëm" # Il est évanoui print(predict_wolof_with_logistic_regression(text)) print(predict_sentiment_svm(text)) >>> NEGATIVE >>> NEGATIVE ``` ##### meth 2 > by calling a script that does all the work for us ```py text = "Na nga def ?" var = !python model/wolof_sentiment.py $text print(var[-1]) >>> 1 ``` - 💪 Model performance: Here are the results …