khalidiktib/morocco-weather-platform

# 🇲🇦 Morocco Weather Intelligence Platform ### End-to-End Automated Data & ML Pipeline An automated production-grade pipeline that ingests real-time weather data for Moroccan cities, transforms it through a multi-layer Snowflake data warehouse, and generates daily temperature forecasts using a Prophet time-series model — all orchestrated with Apache Airflow running inside Docker. --- ## Architecture ``` Open-Meteo API (6 Moroccan Cities) │ ▼ [Airflow DAG — Daily] │ ▼ Snowflake — RAW Layer (raw hourly weather data) │ ▼ [dbt Run] │ ▼ Snowflake — STAGING Layer Snowflake — MART Layer (cleaned, enriched view) (daily aggregations table) │ ▼ [Prophet ML Model] │ ▼ Snowflake — MART Layer (7-day forecasts per city) ``` --- ## Tech Stack | Layer | Technology | |---|---| | Ingestion | Python, Open-Meteo API | | Orchestration | Apache Airflow | | Containerization | Docker, Docker Compose | | Data Warehouse | Snowflake | | Transformation | dbt (data build tool) | | Forecasting | Prophet (Meta) | | Language | Python, SQL | --- ## Data Pipeline ### RAW Layer Raw hourly weather data landed directly from the Open-Meteo API for 6 Moroccan cities: **Casablanca, Rabat, Marrakech, Fes, Agadir, Tangier**. Variables ingested per city per hour: - Temperature (°C) - Relative Humidity (%) - Wind Speed (km/h) - Precipitation (mm) - Weather Code ### STAGING Layer dbt view that cleans and enriches raw data: - Extracts date and hour from timestamp - Maps weather codes to human-readable descriptions (Clear sky, Rain, Thunderstorm, etc.) - Renames and standardizes columns ### MART Layer Two tables produced by dbt and the ML model: **`MART_WEATHER_DAILY`** — Daily aggregations per city: - Avg / Min / Max temperature - Avg humidity and wind speed - Total precipitation - Dominant weather condition **`WEATHER_FORECASTS`** — Prophet model output: - 7-day temperature forecast per city - Confidence intervals (lower/upper bound) - Training timestamp --- ## Automation A sin …