Kona15/yoruba-news-classifier

# Yoruba Text Classification Model for News Categorization ## Project Overview This project develops a machine learning model to classify Yoruba news articles into five categories: Politics, Sports, Entertainment, Health, and Business. The model uses TF-IDF (Term Frequency-Inverse Document Frequency) for feature extraction and Logistic Regression for classification. ## Project Structure \`\`\` ├── scripts/ │ ├── combine_and_train.py # Trains with combined dataset (5100 samples) │ ├── train_model.py # Original training script │ └── predict.py # Makes predictions on new text ├── notebooks/ │ └── yoruba_text_classification.ipynb # Jupyter notebook with analysis ├── data/ │ ├── yoruba_news_dataset.csv # 100 original Yoruba samples │ └── yoruba_v0_boost_dataset.csv # 5000 boost dataset samples ├── models/ │ ├── model.pkl # Trained Logistic Regression model │ ├── vectorizer.pkl # TF-IDF vectorizer │ └── metadata.json # Model metadata and statistics ├── app/ │ ├── page.tsx # Main web interface │ └── api/predict/route.ts # API endpoint for predictions └── components/ └── prediction-result.tsx # Result display component \`\`\` ## Dataset ### Combined Training Data - **Total Size**: 5,100 authentic Yoruba news samples - **Components**: - Original dataset: 100 samples (20 per category) - Boost dataset: 5,000 samples (1,000 per category) - **Categories**: Politics, Sports, Entertainment, Health, Business - **Format**: CSV with text, category, and optional english_translation - **Language**: Yoruba with proper diacritical marks preserved ### Category Distribution - Politics: 1,020 samples - Sports: 1,020 samples - Entertainment: 1,020 samples - Health: 1,020 samples - Business: 1,020 samples ## Methodology ### 1. Feature Extraction - **TF-IDF Vectorizer**: Converts text into numerical features - Max features: 500 (increased from 150 for better capture) - N-g …