AremuAdeolaJr/An-NLP-Deep-Dive-into-Yoruba-English-Code-Switching

# An NLP Deep Dive into Yoruba-English Code-Switching: Synthetic Corpus Construction, Human Quality Review, and Automated Annotation This research outlines a methodology for constructing a synthetic, annotated corpus focused on **Yoruba-English code-switching**, a linguistic phenomenon prevalent in multilingual Nigerian discourse. Utilizing a pipeline integrated with **GPT-4** and **LangChain**, the study addresses the chronic scarcity of high-quality data for low-resource language pairs. The framework employs thematic constraints and **UMAP-based semantic analysis** to ensure generative diversity, followed by a rigorous human-in-the-loop review to mitigate model errors, such as diacritic omission and syntactic inaccuracies. While automated **Part-of-Speech (POS)** and **Named Entity Recognition (NER)** layers provide structural depth, the study identifies significant limitations in existing language identification models like **FastText** for token-level classification. Ultimately, the project offers a scalable, reusable architecture for developing linguistically meaningful datasets, advocating for NLP systems that accurately reflect the fluid communicative realities of under-represented bilingual communities. # Data The code generated 50 sentences acorss domain but the 35 were used in the analysis and saved as /approved_sentences.csv # license: cc-by-nc-4.0 # Languages There are 2 languages available : English (en) and Yorùbá (yo). # Link to Meduim Article Post medium.com