SaadNamoune/MedLLM-Algerian

# MedLLM-Algerian **Medical Large Language Model Fine-Tuning for the Algerian Healthcare Context** — Arabic/French bilingual LLM training pipeline covering PT → SFT → RLHF → DPO/ORPO stages, adapted for Algerian medical terminology and clinical workflows. --- ## Overview MedLLM-Algerian fine-tunes open-source LLMs on Arabic/French medical corpora, with specific adaptations for: - **Algerian medical Arabic** (Darija + MSA clinical terminology) - **French-Arabic code-switching** common in Algerian medical records - **Local disease prevalence** (diabetes, hypertension, leishmaniasis, viral hepatitis) - **SANTE platform integration** (Algeria's national electronic health record system) The training pipeline supports four stages progressively: ``` Pre-training (PT) → Supervised Fine-Tuning (SFT) → RLHF → DPO/ORPO alignment ``` --- ## Training Pipeline ``` data/pretrain/ Arabic medical articles + French clinical texts │ ▼ training/pt_train.py Pre-trained base (continued pretraining on medical domain) │ ▼ training/sft_train.py (LoRA / QLoRA) Instruction-tuned model (Q&A, diagnosis assistance) │ ▼ training/rlhf_train.py RLHF reward model + PPO alignment │ ▼ training/dpo_train.py or training/orpo_train.py Final aligned model (safe, factual, bilingual) ``` --- ## Supported Base Models | Model | Parameters | Language | Medical Adaption | |---|---|---|---| | Llama-3.1-8B | 8B | EN/AR | Via continued PT | | Mistral-7B-v0.3 | 7B | EN/FR | Via SFT | | Jais-13B | 13B | AR/EN | Natively Arabic | | Qwen2.5-7B | 7B | ZH/EN/AR | Multilingual | --- ## Dataset Strategy | Stage | Data | Size | |---|---|---| | PT | Arabic medical Wikipedia, PubMed abstracts (FR) | ~500MB | | SFT | Medical QA pairs (Arabic + French), clinical notes | ~50K pairs | | RLHF | Human-ranked response pairs from Algerian physicians | ~5K pairs | | DPO | Preference pairs: safe/unsafe responses | ~10K pairs | --- ## Quick Start ```bash git clone github.com …