onwunaru760-a11y/scd-phenotype-clustering-nlp

Unsupervised SCD phenotype discovery using Bio+Clinical BERT, UMAP, and HDBSCAN — calibrated for the African disease context # SCD Phenotype Clustering via Clinical NLP > **Unsupervised discovery of clinically meaningful patient subgroups in Sickle Cell Disease using Bio+Clinical BERT embeddings, UMAP dimensionality reduction, and HDBSCAN clustering — calibrated for the African disease context.** --- ## Overview Sickle Cell Disease (SCD) is not one disease. It presents differently depending on geography, comorbidities, access to care, and age — yet most NLP research on SCD uses datasets and assumptions that don't reflect the African patient. This project uses unsupervised machine learning on publicly available clinical text (PMC-Patients dataset) to identify latent phenotypic subgroups in SCD patients, with five research questions anchored in the realities of low-resource settings. This is not an academic exercise. The goal is to generate cluster-level insights that could eventually inform triage decisions, referral protocols, and health system design in places where specialist care is scarce. --- ## Research Questions | # | Question | |---|----------| | 1 | Do SCD patients cluster into geographically distinct phenotypic subgroups based on clinical text? | | 2 | Does malaria co-infection create a distinguishable clinical subgroup? | | 3 | Are patients with delayed presentation to care phenotypically distinguishable from those who present early? | | 4 | Do paediatric and adult SCD patients separate into distinct clusters without explicit age labelling? | | 5 | Does treatment access (or lack thereof) emerge as a clustering signal in clinical documentation? | --- ## Pipeline Architecture ``` PMC-Patients Dataset │ ▼ [01_load_and_filter.py] ← Filter to SCD-relevant patient records │ ▼ [02_preprocess.py] ← Clinical text cleaning, normalisation │ ▼ [03_embed_bioclinicalbert.py] ← Bio+Clinical BERT sentence embeddings │ ▼ [04_tfidf_baseline.py] ← TF-IDF/LSA baseline for comparison │ ▼ [05_umap_reduce.py] ← UMAP dimensionality reduction (2D/3D) │ ▼ [06 …