maroxtn/tun-sentiment

My solution in Zindi Tunisian Sentiment Analysis competition. Ranked #1st. # Tunisian Arabizi Sentiment Analysis Zindi Competition (Ranked #1st): This repo contains my approach in solving the Tunisian dialect sentiment analysis problem. My solution took place in Tunisian Sentiment Analysis competition in which I ranked first before getting disqualified (for allegedly breaking a rule). Even after the competition ended, my solution scored the best by a significant margin on both public and private dataset. The competition posed several challenges: 1- Dataset is relatively small 2- Tunisian dialect has a high degree of variance 3- No available pretrained models for Tunisian dialect nor any Maghrebi dialect 4- The text is in latin characters (Arabizi) not Arabic ---- ## My solution Due to the small size of the dataset it would deem impractical to train a model from scratch; it would be hard for a model to learn a language with as many specific cases as Tunisian with a dataset only containing 70k sentences. Appropriately, using a pretrained model seemed like the natural solution. If we analyze the Tunisian dialect, we would find that it is a melting pot of many languages: Arabic by a big margin (root of words, grammar, etc ...), Amazigh words (which is shared among all Maghrebi dialects), French (also shared with some Maghrebi dialects), and to a certain extent some English and Italian. Because of this diversity, using an ensemble of multiple pretrained language models fine-tuned on the same dataset would surely yield higher accuracy since every model would contribute to the language understanding by a bit. But how is it possible to take advantage of Arabic pretrained model and the dataset is in latin letters, one might ask. For that reason, I trained an independent transformer model for transliterating from Latin letters to Tunisian using a dataset that I personally harvested and annotated. This dataset contains around 17k commonly used Tunisian words in both Arabic letters and Arabizi. More on the dataset later. After trying se …