djabelo712/ml-supervised-learning

Supervised learning from scratch and with Scikit-learn: polynomial regression, regularization, cross-validation, logistic regression, SVM, KNN and decision trees , AIMS Ghana 2025 # Applied Machine Learning: Regression, Classification and Model Selection **AIMS Ghana — 2025** *Ounimborbitibou Djabon* --- ## Overview This repository contains two hands-on machine learning projects completed as part of the MSc in Mathematical Sciences at the African Institute for Mathematical Sciences (AIMS Ghana) in 2025. Together, they cover the two fundamental pillars of supervised learning: **regression** (fitting continuous data) and **classification** (predicting discrete categories). All implementations are done in Python using NumPy, Matplotlib, and Scikit-learn, with a strong emphasis on understanding what happens mathematically under the hood before reaching for high-level library calls. --- ## Repository Structure ``` ml-supervised-learning/ │ ├── notebooks/ │ ├── 01_regression_model_selection.ipynb # Polynomial regression, basis functions, regularization, cross-validation │ └── 02_classification.ipynb # Logistic regression, SVM, KNN, Decision Tree │ ├── data/ │ ├── gabor_data.out # 1D dataset used for regression experiments │ ├── data.out # Secondary regression dataset │ └── ex2data1.txt # University admissions dataset for classification │ ├── images/ # Figures exported from notebooks (optional) │ ├── .gitignore └── README.md ``` --- ## Project 1 — Regression and Model Selection **Notebook:** `notebooks/01_regression_model_selection.ipynb` ### Problem Given a noisy 1D dataset (`gabor_data.out`), the goal is to find the model that best explains the data — without overfitting or underfitting. This is a classic problem at the heart of statistical learning theory. ### Methods Implemented #### 1. Polynomial Regression - Built the Vandermonde design matrix from scratch using NumPy - Solved the normal equations analytically: $\hat{\theta} = (X^T X)^{-1} X^T y$ - Compared polynomial degrees from 2 to 22 and observed overfitting behaviour #### 2. Train/Validation Split - …