HabtamuFeyera/oromo_chatbot_finetune

# Afaan Oromoo Passport Chatbot - Fine-tuning Guide ## 📋 Overview This project provides a complete pipeline for fine-tuning a small language model to create a chatbot that answers questions about Ethiopian passport services in **Afaan Oromoo** (Oromo language). ## 📊 Dataset Summary | Metric | Value | |--------|-------| | **Total Samples** | 392 | | **Training Samples** | 352 | | **Validation Samples** | 40 | | **Language** | Afaan Oromoo | | **Domain** | Ethiopian Passport Services | ## 🗂️ Project Structure ``` oromo_chatbot_finetune/ ├── prepare_data.py # Data preparation script ├── train.py # Fine-tuning script (QLoRA) ├── inference.py # Testing and inference script ├── requirements.txt # Python dependencies ├── README.md # This file ├── data/ # Prepared training data │ ├── train_instruction.jsonl │ ├── train_chat.jsonl │ ├── train_alpaca.jsonl │ ├── val_instruction.jsonl │ ├── val_chat.jsonl │ └── val_alpaca.jsonl └── model/ # Fine-tuned model (after training) ``` ## 🚀 Quick Start ### Step 1: Install Dependencies ```bash pip install -r requirements.txt ``` ### Step 2: Prepare Data (Already Done) ```bash python prepare_data.py ``` ### Step 3: Fine-tune the Model ```bash # Basic training with default settings (Qwen2.5-3B) python train.py # Custom configuration python train.py \ --model_name Qwen/Qwen2.5-3B-Instruct \ --epochs 3 \ --batch_size 4 \ --learning_rate 2e-4 \ --lora_r 16 ``` ### Step 4: Test the Model ```bash python inference.py --model_path ./model ``` ## 🖥️ Hardware Requirements | Configuration | Minimum GPU | Recommended GPU | |---------------|-------------|-----------------| | **QLoRA (3B)** | 8GB VRAM | 12GB+ VRAM | | **QLoRA (7B)** | 12GB VRAM | 16GB+ VRAM | | **Full LoRA (3B)** | 16GB VRAM | 24GB+ VRAM | **Budget Options:** - Google Colab Free (T4 - 16GB) ✅ Works for QLoRA with 3B models - Google Colab Pro (A100) - Faster training - RunPod / Lambda L …