kiggundusharif-v1/nphc2024-dropout-prediction

MSc: predicting school dropout (ages 6-17) from the Uganda NPHC 2024 census - 8 models, stacking ensemble, distillation/quantization, SHAP/LIME explainability, and a Streamlit early-warning app that loads its models from Kaggle at runtime. # Predicting School Dropout — Uganda NPHC 2024 End-to-end pipeline: integrate the census Population + Household micro-data, engineer a leak-free feature set, train and compare **8 models** across three families, build an ensemble, compress it, explain it, and deploy an interactive early-warning app. Experiments are tracked in **MLflow**. ## Project structure ``` data/ # all parquet (zstd) — see KAGGLE.md for which file does what raw_population.parquet # 4.69M persons, selected columns (pre-merge) raw_household.parquet # 1.07M households, selected columns (pre-merge) merged_raw.parquet # person↔household merge (schooling-age window) cleaned_full.parquet # 1.19M target-base rows, 35 features + target sample_{train,val,test}.parquet # grouped, stratified working sample (~60k) src/ paths.py # ONE place resolving data/output dirs (local · Kaggle · Colab) config.py # ONE place holding every hyper-parameter (NPHC_* env overrides) extract_merge.py # STEP 0: read raw parquet (or .dta), merge on HH_CASEUID features.py # cleaning + feature engineering + target (shared) data_prep.py # cleaned dataset, grouped stratified split, SMOTE, scaler graph_build.py # PyG graph (household + parish edges) for the GNNs gen_notebook.py # builds the EDA notebook gen_kaggle_notebook.py # builds notebooks/kaggle_run_all.ipynb make_kaggle_package.py # builds the Kaggle upload zip (converts data to zstd parquet) kaggle_models.py # fetches trained artefacts from Kaggle at runtime (see DEPLOY.md) make_app_defaults.py # freezes the app's feature medians -> app/feature_defaults.json notebooks/ 01_data_integration_cleaning_eda.ipynb # documented cleaning + Plotly EDA + splits experiments/ # one file per model, all log to MLflow + reports/experiments/ _common.py _gnn_common.py exp_01_random_forest. …