fruStoic/kmeans-soil-clustering

K-Means clustering of African soil chemistry data from the AfSIS open dataset to identify soil fertility archetypes. # K-Means Clustering of AfSIS Soil Chemistry Data This project implements a K-Means clustering application using soil chemistry data from the Africa Soil Information Service (AfSIS), available through the Registry of Open Data on AWS. The project was developed for BAN6440 Machine Learning. Its goal is to identify meaningful groupings in soil chemistry data without using predefined soil labels. ## Dataset The project uses public AfSIS datasets containing wet-chemistry and georeference information. The main CROPNUTS dataset contains 1,907 soil samples. The clustering model uses 17 numeric chemistry features: - EC - ExAc - ExBas - M3 Al - M3 B - M3 Ca - M3 Cu - M3 Fe - M3 K - M3 Mg - M3 Mn - M3 Na - M3 P - M3 S - M3 Zn - PH - PSI The source CSV files are downloaded automatically on first run and cached in the local `data/` directory. ## Project Structure ```text kmeans_soil/ ├── src/ │ ├── data_loader.py │ ├── preprocessing.py │ ├── clustering.py │ └── main.py ├── tests/ │ └── test_pipeline.py ├── data/ ├── outputs/ ├── requirements.txt ├── .gitignore └── README.md ``` ## Method The application follows five main steps: 1. Download and merge the AfSIS datasets using `SSN`. 2. Select the 17 soil chemistry features. 3. Preprocess the data by: - clipping negative below-detection values to zero; - imputing missing values with the median; - applying `log1p` to strongly skewed features; - standardizing all features with `StandardScaler`. 4. Evaluate K-Means models for `k = 2` through `k = 10` using: - the elbow method; - silhouette score. 5. Fit the final K-Means model and profile the clusters using chemistry, country, and soil depth. ## Results The highest silhouette score was obtained at: ```text k = 2 Silhouette score = 0.244 ``` The final model produced: | Cluster | Samples | General profile | |---|---:|---| | Cluster 0 | 1,195 | Lower-base, more acidic | | Cluster 1 | 712 | Higher-base, less acidic | Cluster 0 had an average pH of approximatel …