DEVELOPMENT OF A REGIONAL SYNONYM NORMALIZATION FRAMEWORK FOR NIGERIA AND GHANA USING SENTENCE EMBEDDINGS, HDBSCAN CLUSTERING, AND LLM ENHANCEMENT

Artificial Intelligence (AI) and Natural Language Processing (NLP) are also increasingly being implemented in Africa, but the linguistic diversity in the continent, especially the use of synonyms by different sections of the continent, pose some challenges. These variations can result in semantic errors in Nigeria and Ghana countries, and affect the performance of the model, especially for speakers of a particular language. This work is designed to improve the Large Language Models (LLMs) by concentrating on identifying and normalizing regional synonyms, applying a framework that includes generating semantic embeddings, training existing models for domain alignment, and assessing the performance improvements. The study collects and preprocesses the regional vocabulary, applies semantic embedding methods and clusters the data using HDBSCAN, which is an unsupervised clustering method. Model performance is evaluated using criteria like accuracy, recall. The fine-tuned model shows significant improvements in memory, clustering cohesiveness, and semantic retrieval effectiveness, with a Silhouette Score improvement of 62 points, which points to more reliable clustering and less ambiguity. In summary, fine-tuning and synonym normalization for a specific domain and language can greatly help both NLP models to achieve a more inclusive AI solution for African languages in general.