Odhiambo-nelson/Swahili-and-English-Swahili-hate-speech-models-and-datasets

# Swahili-and-English-Swahili-hate-speech-models-and-datasets SwahBERT Fine-Tuning Overview This repository contains scripts and configurations for fine-tuning the SwahBERT model, a BERT-based language model optimized for Swahili texts. The fine-tuning process adapts SwahBERT to specific hate speech detection using dataset having Swahili and code-switched English-Swahili textual data. The model could be fine-tuned for other specific task using labeled dataset, such as text classification, named entity recognition, or sentiment analysis. Prerequisites Before running the fine-tuning script, ensure you have the following installed: • Python 3.8+ • PyTorch • Transformers (Hugging Face) • Datasets (Hugging Face) • scikit-learn • pandas • tqdm • CUDA (if training on a GPU) Install dependencies using: pip install torch transformers datasets scikit-learn pandas tqdm For GPU acceleration, ensure you have the correct CUDA version installed and verify it with: python -c "import torch; print(torch.cuda.is_available())" Dataset Preparation Ensure your dataset is formatted as a CSV or JSON file with labeled text samples. The dataset should have the following format: Text label Sample Swahili sentence1 0 Sample Swahili sentence 2 1 Modify the fine-tuning script to load the dataset correctly: from datasets import load_dataset dataset = load_dataset("csv", data_files={"train": "path/to/dataset.csv", "test": "path/to/test_dataset.csv"}) Fine-Tuning Run the fine-tuning script with the following command: python fine_tune_swabert.py --train_data path/to/dataset.csv --test_data path/to/test_dataset.csv --epochs 3 --batch_size 16 --learning_rate 5e-5 --output_dir saved_model/ Customizing Hyperparameters You can modify training parameters such as: • --epochs: Number of training epochs (default: 3) • --batch_size: Training batch size (default: 16) • --learning_rate: Learning rate for the optimizer (default: 5e-5) • --max_seq_length: Maximum token length per input text (default: 5 …