KoredeAkande/nigerian_isp_sentiment_analysis

Minerva Capstone Project: Conducting an Aspect-Based Sentiment Analysis on ISPs in Lagos, Nigeria as a means to evaluate the feasibility of using social media as a Decision Support System Nigerian ISP Aspect-Based Sentiment Analysis ============================== This repository contains the data and code utilized in my undergraduate thesis, where I conduct an (aspect-based) sentiment analysis of Internet Service Providers in Lagos, Nigeria, using Twitter data. The project culminates in a Tableau dashboard to inform understanding of customers and facilitate strategic decision making. Skip to the `Repository Organization` below for an overview of the project structure. Sentiment Analysis ------------ Three models were experimented with for Nigerian Internet Service Providers' sentiment analysis, based on two factors: *Multilingualism* and *Proximity to Problem Domain (i.e. Twitter)*. The specific models fine-tuned (and their justification) include: - BERTweet (Proximity to Problem Domain) - Multilingual-BERT (Multilingualism) - XLM-roBERTa-base (Multilingualism & Proximity to Problem Domain) After experimenting with reweighting the loss function, increasing the batch size, and oversampling the minority to account for class imbalance, and experimenting with weight decay to address overfitting, the best checkpoints for the different models obtained the following validation results | Model | Accuracy | Precision | Recall | F-1 | |:----------------:|:------------:|:---------:|:------:|:-----:| | M-BERT | 65.8% | 59.8% | 49.9% | 51.8% | | BERTweet | **85.5%** | 81.9% | **84.3%** | **83.0%** | | XLM-roBERTa-base | 82.9% | **87.3%** | 72.4% | 77.0% | **Note:** All metrics above are macro-averaged Aspect-Based Sentiment Analysis (ABSA) ------------ ABSA can be broken down into two subtasks: Aspect Extraction (AE) and Aspect Sentiment Classification (ASC) #### Aspect Extraction (AE) The aspect extraction subtask was framed as a multi-label classification problem. Hence, a single tweet can have multiple aspects (in our case one or more out of *price, speed, coverage, customer service*, and …