Okoth67/wekeza-llm-finetuning

Wekeza LLM is an open-source financial assistant focused on the Kenyan investment ecosystem. This project fine-tunes the Mistral-7B-Instruct model using QLoRA on a custom Alpaca-style dataset to help users answer questions about money market funds and other investment options in Kenya. # Wekeza LLM: Fine-Tuning Mistral-7B for Kenyan Investment Advice 🇰🇪💰 Wekeza LLM is a language model fine-tuned to provide intelligent answers about money market funds and investment options in Kenya. ## 🚀 Project Goal Build a specialized financial assistant using QLoRA and instruction fine-tuning techniques, designed to: - Understand investment-related questions in Kenya. - Answer in a helpful, local, and practical tone. - Support formats like Alpaca and ChatML for extensibility. ## 🛠️ Tech Stack - Model: `mistralai/Mistral-7B-Instruct-v0.1` - Fine-tuning: QLoRA (via `peft`) - Tokenization: `transformers` library - Dataset: Custom curated in Alpaca format (JSONL)