Skywalkingzulu1/impilo-llm

On-device isiZulu/English health LLM for rural SA clinics — ADTC 2026 Laptop LLM Challenge # Impilo Health LLM **On-device isiZulu/English health LLM for rural South African clinics.** A compact, quantized language model (2.6B params, GGUF Q4_K_M) that runs entirely offline on 8 GB RAM laptops — delivering clinical triage guidance and patient education in isiZulu and English for community health workers in rural South Africa. --- ## Problem South Africa's 4,000+ primary healthcare facilities serve 50M+ people. Community health workers (CHWs) use basic 8 GB RAM laptops with no reliable internet. Cloud AI is inaccessible due to cost and connectivity. Patients speak isiZulu; clinical resources are English-only. Printed reference materials are outdated. **Impilo fills the gap:** a health LLM that fits on the hardware already in clinics, speaks the local language, and works without internet. ## Quick Start ```bash # 1. Download the model (requires curl or wget) bash download_model.sh # 2. Run inference with llama.cpp ./llama-cli -m model/impilo-health-2b-q4_k_m.gguf \ -p "A 45-year-old patient presents with chest pain and shortness of breath. What should the CHW do?" \ -n 256 ``` ## Submission Structure | File | Purpose | |------|---------| | `metadata.json` | Team, model, and test prompt metadata | | `download_model.sh` | Downloads the GGUF weight file | | `REPORT.md` | Full technical writeup (judges read this) | | `training/` | QLoRA fine-tuning pipeline | | `evaluation/` | Accuracy and performance benchmarks | | `pipeline/` | Quantization and deployment scripts | | `docs/` | Architecture, dataset, benchmarks, demo script | ## Model | Property | Value | |----------|-------| | Base | Gemma 2B (Google) | | Quantization | GGUF Q4_K_M | | Size | ~1.5 GB | | RAM usage | ~3.2 GB | | Speed | ~18 tok/s (8 GB laptop) | | Languages | English, isiZulu | | Domain | Clinical triage, patient education | ## Training ```bash pip install -r training/requirements.txt python training/prepare_dataset.py python training/train.py --config training/config.yaml ba …