danielochieng23/nigeria-dhs-malaria-under5-prediction

Cost-sensitive XGBoost pipeline for under-five malaria risk (Nigeria 2024 DHS) # Under-Five Malaria Risk Prediction — Nigeria 2024 DHS A cost-sensitive, survey-aware machine-learning pipeline that predicts malaria positivity in children under five, with the explicit goal of **minimising dangerous missed cases** (false negatives) under realistic survey constraints. > ⚠️ **Critical data limitation — read first.** > The biomarker malaria test result (DHS `hml32` microscopy / `hml35` RDT) lives > in the **Household Member (PR) recode**, which was **not** included in the > supplied data (only the BR, IR and KR recodes are present). The pipeline > therefore runs on a **clearly-labelled proxy outcome** — `h71` *"was the > respondent told the child had malaria"*, restricted to children with a recent > febrile episode. This is a **caregiver-/provider-reported diagnosis, not a > laboratory test**, and is subject to recall and care-seeking bias. To deliver > the analysis specified in the brief, **add `NGPR8BFL.dta` and set > `OUTCOME_MODE = "biomarker"`** in `src/config.py` (the biomarker merge is the > documented next step). --- ## What the pipeline produces | Stage | Module | Key outputs | |------|--------|-------------| | Data prep | `src/data_prep.py` | Cleaned analytic dataset, data dictionary, exclusion flow | | Data quality | `src/data_quality.py` | Missingness, constants, cardinality, outliers, correlation, imbalance + recommendations | | EDA | `src/eda.py` | Survey-weighted prevalence (design-based 95% CI), summary stats, distribution & correlation figures | | Modeling | `src/modeling.py` | Logistic benchmark, standard XGBoost, cost-sensitive XGBoost (repeated stratified CV + tuning, locked test) | | Threshold | `src/threshold.py` | Cost-optimal decision threshold (FN-weighted) + isotonic calibration | | Evaluation | `src/evaluate.py` | Clinically-aligned metrics, ROC/PR, confusion matrices, calibration, cluster-aware sensitivity | | Interpretation | `src/interpret.py` | SHAP summary + gain importance | | Report | `src/report.py` | Self-con …