hermitquant/nigeria-hospital-patients-ml

# Nigeria Hospital Patients ML Project End-to-end preprocessing and EDA pipeline for the dataset: hf://datasets/electricsheepafrica/Nigeria-hospital-patients/patients.parquet This project loads the dataset directly from Hugging Face using fsspec, reports basic statistics, performs exploratory data analysis with visualizations, and builds a reusable scikit-learn preprocessing pipeline. Artifacts and figures are saved to dedicated folders. ## Dataset structure - patient_id (object): Unique patient identifier. - name (object): Patient name. - age (int64): Age in years; 0 may represent newborns. - arrival_date (datetime64[ns]): Admission/arrival timestamp. - departure_date (datetime64[ns]): Discharge/departure timestamp. - service (object): Clinical service/department (e.g., surgery, emergency, ICU, general_medicine). - satisfaction (int64): Patient satisfaction score (observed 60–99). ## Features - Loads parquet from Hugging Face `hf://` URLs via fsspec - Prints and saves dataset schema, missingness, numeric and categorical summaries - Visualizations: histograms, boxplots, categorical counts, correlation heatmap, missingness bars, optional time-series counts if a datetime column exists - Type inference for numeric/categorical/datetime columns (with safe, explainable logic) - Robust preprocessing pipeline using scikit-learn: - Numeric: impute (median) + scale (StandardScaler) - Categorical: impute (most_frequent) for missing values + OneHotEncoder (handles unknowns) - Datetime: optional extraction of calendar features (year, month, day, dow) - Pipeline persisted with joblib ## Project Structure - main.py - src/ - preprocessing_utils.py - reports/ - figures/eda/ - artifacts/ - data/ - processed/ ## Requirements - Python 3.10+ Install dependencies: ```bash pip install -r requirements.txt ``` ## Usage Basic run with defaults (uses the specified HF dataset): ```bash python main.py \ --hf-path hf://datasets/electricsheepafrica/Nigeria-hospital-patients/patients.pa …