ThatDawg/kiswahili-ai

Swahili instruction tuned model - AutoScientist Challenge 2026 # Kiswahili AI — Swahili Instruction Model > AutoScientist Challenge 2026 — Language Category ## Overview Kiswahili AI is a Swahili instruction-tuned language model fine-tuned from **Llama-4-Scout-17B-16E-Instruct (109B MoE)** using **AutoScientist** by Adaption Labs. It combines **4 public Swahili datasets** into a unified instruction dataset (~52K rows), processes them through **Adaptive Data** for quality enhancement, and trains via AutoScientist's closed-loop co-optimization. **Result**: **73% win rate** (adapted) vs **27%** (baseline) — **+170% relative improvement** on Adaption's held-out test set. **Why Swahili?** Over 100 million speakers across East Africa. Swahili is a low-resource language where current LLMs show a 28-45% performance gap compared to English. ## Dataset | Source | Rows | Type | |--------|:----:|------| | FineTome-20k-sw | 17,982 | General instruction | | KenSwQuAD | 7,506 | Extractive QA | | Code-170k-swahili | 14,969 | Code conversations | | Swahili-Corpus-Dataset | 12,267 | Raw text (converted to instruction) | | **Total** | **52,118** (after dedup) | | ### Data Adaptation The raw dataset scored **Grade D (6.9th percentile)**. After Adaptive Data processing (deduplication + reasoning traces + hallucination mitigation), quality improved **+62% to Grade B (25.6th percentile)**. ## Training | Setting | Value | |---------|-------| | **Base Model** | meta-llama/Llama-4-Scout-17B-16E-Instruct (109B MoE) | | **Method** | SFT with LoRA (r=64, alpha=128, all-linear) | | **Epochs** | 1 | | **Batch Size** | max | | **Learning Rate** | 0.0001 (cosine scheduler) | | **Warmup Ratio** | 0.03 | | **Weight Decay** | 0.02 | | **Platform** | AutoScientist by Adaption Labs | ### Results | Metric | Baseline | Adapted | Improvement | |--------|:--------:|:-------:|:-----------:| | Win Rate | 27% | 73% | **+170%** | | General Category Win Rate | 31% | 69% | **+123%** | ## Pipeline ``` Public HF Datasets → Blend Script → JSONL → Adaptive Data → A …