Phylisss-Shamiso/Shona-News-Classification-

NLP for low resource languages using contrastive and transfer learning. # Shona-News-Classification- NLP for low resource languages using contrastive and transfer learning. Abstract-Shona is a language spoken by over 10 million people in Zimbabwe, however very little research exists as to how it can also be used to train models with high accuracy as it does not have a large electronic corpus. While a lot of ground breaking researches in the field of Natural Language Processing have led to the creation of large language and pre-trained models, that are created using very large datasets, languages such as Shona that have very little electronic archives compared to other mainstream languages like English, have always been left behind. This research aims to address this challenge by introducing a new model that uses a dynamic margin based triplet loss function with a lambda parameter that focuses on understanding the underlying relationships in the data in order to separate articles that belong in different classes based on their context. This research uses the MasakhaNEWS [1] dataset with 369 articles that are split into four categories. Through transfer learning, the model that this research produced was used to generate embeddings for each of the 4 categories on a separate dataset which were used as input to a classification algorithm. A comparison with the base model that was trained on a lot of data, on the same dataset showed that accuracy improved by 14% along other metrics. Based on the results of this model, this research concluded that a model does not need a lot of training datasets in order to learn and classify text data but rather, it requires techniques that will focus on truly comprehending the underlying relationships in a dataset based on the context, in situations where a large dataset is not available.