NembotJules/Cameroon-Air-Quality-Prediction

End to End Mlops System to predict the air quality in Cameroon. Everything is automatic from the Data pipeline, to model retraining and redeployment. # **Air Quality Prediction System for Cameroon** This repository hosts the codebase for an end-to-end machine learning system designed to predict air quality across the ten major cities of Cameroon. By leveraging satellite data, weather information, and advanced machine learning workflows, this project eliminates the need for unreliable ground-based sensors in resource-constrained settings. The system is fully autonomous, and get better over time with automatic model retraining and redeployment when drift is detected. # Features 1. **End-to-End Architecture**: - Data Pipeline: Automates daily data fetching, transformation, and storage using Prefect. - Offline Components: Includes EDA, model development, and a robust model registry with MLflow. - CI/CD Pipeline: Automates testing, building Docker images, and deployment to AWS ECS using GitHub Actions. - Monitoring Pipeline: Tracks data drift using Evidently, sends alerts, and integrates seamlessly with the data pipeline. - Model Retraining Pipeline: Retrains the model weekly, compares performance, and redeploys only if the new model outperforms the production model. 2. **Scalable Deployment**: - Hosted on AWS ECS with containerized model APIs for high availability. - Integrated with a Load Balancer to ensure stable traffic routing despite dynamic IP changes. 3. User-Friendly Interface: - Streamlit Cloud host an interactive UI for visualizing air quality predictions and insights. # System Architecture The system is composed of the following key components: - **Data Pipeline**: Runs daily at 01:00 AM using Prefect. Fetches weather and AQI data for ten cities, merges, transforms, and stores historical data for retraining. - **Offline Development**: EDA and Preprocessing: Comprehensive exploratory data analysis and feature engineering. Model Training: Uses MLflow for experiment tracking and maintains a version-controlled model registry. - **CI/CD Pipeline**: Ensures seamless deployment through GitHu …