DG-AOG/yoruba_response

# yoruba_response ## Yoruba Text Preprocessing ## Project Overview This project preprocesses a Yoruba text dataset for basic natural language understanding tasks like word and sentence counts, as well as frequency analysis. ## Objective To clean and analyze Yoruba text data for potential use in language modeling or chatbot training. ## Dataset Source - **Name:** Dolly-HH-RLHF Yoruba Response Dataset - **Link:** Hugging Face Dataset - **Format:** CSV (extracted using Hugging Face SQL Console) ## Preprocessing Summary - Removed punctuation (but kept Yoruba characters) - Normalized whitespaces - Calculated word and sentence counts - Performed word frequency analysis ## Output - Total words: `29,672` - Total sentences: `1,363` - Top 20 most common Yoruba words with frequency in tabular form ## Google Colab Notebook Open in Colab