opensulfware/africa-data-intelligence

Africa Data Intelligence (ADI) is an open-source initiative building reusable infrastructure for collecting, structuring, validating, and exposing publicly available data about Africa. # Africa Data Intelligence ## Open-source data infrastructure for Africa. Africa Data Intelligence (ADI) is an open-source initiative building reusable infrastructure for collecting, structuring, validating, and exposing publicly available data about Africa. We are starting with labour-market intelligence: building an open, continuously updated dataset of African jobs, occupations, and skills demand. ADI is not limited to labour-market data. The long-term goal is to support multiple African datasets across education, healthcare, agriculture, business, climate, public services, and other domains. Built by contributors. Built for Africa. Open to everyone. --- ## Status **v0.1 — in development.** Milestone 1 (foundation: domain, ports, persistence, migrations). ## Quick start Requires Python 3.12+ and PostgreSQL 14+. No Docker — ADI runs directly on Linux. ```bash make setup # venv + dependencies + .env (first run takes ~10 min) make test # verify it works — needs no database and no network ``` Then `make` on its own lists everything: | Command | What it does | |---|---| | `make setup` | create the venv, install dependencies, write `.env` — **start here** | | `make test` | run the test suite | | `make check` | lint + test — run this before opening a PR | | `make format` | fix what can be fixed automatically | | `make migrate` | apply database migrations (needs PostgreSQL running) | | `make revision M="..."` | autogenerate a migration after editing `models.py` | | `make api` | start the API at 127.0.0.1 | | `make crawl SOURCE= ` | run one source's crawler | | `make sources` | list the vetted sources | | `make clean` | remove the venv and caches | Prefer the raw tools? `make` is a thin wrapper — `pytest`, `ruff check`, `alembic upgrade head` and `scrapy crawl` all work directly inside the venv. ## How it fits together ```text PUBLIC WEB → Scrapy → normalize → validate → dedup → PostgreSQL → REST API ``` ADI is a **modular monolith** …