mulukadan/kenya-agri-llama-finetune

# Kenya Agricultural Advisory Model — QLoRA Fine-Tuning Capstone ## Project Overview This project fine-tunes Meta Llama 3.1 8B Instruct for Kenyan agricultural advisory using QLoRA. The objective was to build a domain-adapted model capable of providing practical, safety-conscious agricultural guidance grounded in Kenyan agricultural sources while avoiding invented pesticide rates, fertilizer rates, veterinary schedules, regulatory requirements, and unsupported guarantees. The final model is referred to as **V5**. --- ## Base Model `meta-llama/Llama-3.1-8B-Instruct` --- ## Final Model Hugging Face repository: `kadanMD/kenya-agri-llama-3.1-8b-v5` The repository contains the merged V5 model. The final LoRA adapter is also preserved separately in: `kenya-agri-llama-adapter-v5/` --- ## Dataset The final V5 training configuration used: - Training examples: 214 - Validation examples: 20 - Test examples: 20 - Format: LLaMA instruction/chat JSONL - Messages per example: - system - user - assistant The dataset was validated for: - JSON structure - correct message ordering - system-prompt consistency - duplicate questions - train/validation/test overlap - tokenizer length constraints The final training set is: `data/train_v3.jsonl` Validation: `data/val.jsonl` Test: `data/test.jsonl` --- ## System Prompt The model was trained with the following system behavior: > You are a Kenyan agricultural advisory assistant. Provide practical, accurate, safety-conscious agricultural guidance grounded in verified Kenyan agricultural sources. Do not invent recommendations, pesticide rates, fertilizer rates, yield guarantees, or government requirements. Where professional or official verification is required, direct the farmer to the appropriate agricultural extension officer, government agency, or qualified professional. --- ## QLoRA Configuration Final training configuration: - LoRA rank: 16 - LoRA alpha: 32 - LoRA dropout: 0.05 - Target modules: - q_proj - …