fzlmn/morocco-data-platform

End-to-end data platform for Moroccan public economic data — Python ETL, PostgreSQL, dbt, and a Streamlit dashboard. # Morocco Data Platform **An end-to-end analytics platform for Moroccan public economic data** — ingestion, tested transformations, and an interactive dashboard. --- ## Why this project Morocco publishes a genuine amount of economic data, but it lives in incompatible places: the World Bank exposes indicators through a JSON API, the HCP releases consumer price indices as spreadsheets, and exchange rates arrive in a third format entirely. Different schemas, different update cadences, no common key. Answering something as simple as *"how has inflation moved relative to the exchange rate over the last decade?"* currently means downloading three files and reconciling them by hand, every time. This project builds the pipeline that does it once, correctly, and repeatably: raw sources land in a warehouse, transformations are version-controlled and tested, and the result is queryable and visualised. --- ## Architecture ```mermaid flowchart LR A[World Bank API] --> D[Extract Python] B[HCP / data.gov.ma CSV] --> D C[FX rates] --> D D --> E[(PostgreSQL raw schema)] E --> F[dbt staging → marts] F --> G[(PostgreSQL analytics schema)] G --> H[Streamlit dashboard] F -.-> I[dbt tests] ``` --- ## Stack | Layer | Tool | Why | |---|---|---| | Ingestion | Python (`requests`, `pandas`) | Handles both API and file sources | | Storage | PostgreSQL | Warehouse for raw and modelled layers | | Transformation | dbt | Version-controlled SQL with built-in testing and lineage | | Visualisation | Streamlit | Fast path from queries to a shareable dashboard | | Environment | Docker Compose | One command to reproduce the whole stack | | CI | GitHub Actions | Data tests run on every push | --- ## Data sources | Source | Content | Format | |---|---|---| | World Bank Open Data | Inflation, GDP, unemployment | JSON API | | HCP / data.gov.ma | Consumer price index | CSV | | Exchange rates | MAD against major currencies | API | All sources are public and openly licensed. No personal data is …