FaresMallouli/zindi-kenya-clinical-llm

Top 10% (19th/400) — Zindi Kenya Clinical Reasoning Challenge. Fine-tuned Flan-T5 with QLoRA to generate clinician-style responses for rural Kenyan healthcare scenarios. # Kenya Clinical Reasoning Challenge - Top 10% Solution This repository contains my solution for the Zindi Kenya Clinical Reasoning Challenge, where I achieved a rank of 19th out of 400 participants, placing in the top 10%. The objective was to develop a model capable of generating clinical responses that match those of real clinicians in rural Kenyan healthcare settings. The solution is built using the `google/flan-t5-base` model, fine-tuned efficiently using QLoRA to meet the strict resource constraints of the competition. ## Table of Contents - The Challenge - My Approach at a Glance - Detailed Methodology - 1. Model Selection - 2. Prompt Engineering - 3. Efficient Fine-Tuning with QLoRA - 4. Finding the Optimal Training Duration with K-Fold CV - 5. Final Model Training: The 600-Step Hypothesis - Code Structure - How to Reproduce - Conclusion & Key Learnings ## The Challenge The core task was to predict a clinician's response to 400 authentic clinical vignettes, each describing a unique medical scenario and a nurse's professional context (experience, facility type, location). This competition was particularly challenging due to: - **A Small, High-Quality Dataset:** Only 400 training samples were provided, making robust validation and overfitting prevention critical. - **Complex, Nuanced Inputs:** The prompts required a deep understanding of medical context, patient symptoms, and resource limitations in rural Kenyan healthcare. - **Strict Resource Restrictions:** The final solution had to be: - Quantized for memory efficiency. - Have an inference time under 100ms per vignette. - Use less than 2 GB of RAM during inference. - Contain a maximum of 1 billion parameters. - The evaluation metric was the **ROUGE Score**. ## My Approach at a Glance My strategy revolved around efficiently adapting a powerful pre-trained model to this specialized, low-data domain while adhering to the tight performance constraints. | Component | My Implementation | Rationale | |- …