tpchiripa/africa-jobs-intel

Turning African job postings into skills-demand intelligence for data careers # africa-jobs-intel **Live dashboard: africa-jobs-intel.onrender.com** African job-market intelligence — extracting skills-demand signals from job postings, starting with South Africa's data/tech job market. What began as manually skimming job adverts for product ideas turned into a validated pipeline covering 9 roles across 6 industries, and a live dashboard showing what employers are actually asking for, with direct links to apply. ## What it does For each tracked role (Data Analyst, Data Engineer, Data Scientist, Software Engineer, Business Analyst, Accountant, Registered Nurse, Civil Engineer, Marketing Specialist), the pipeline: 1. Pulls real job postings from public sources 2. Matches posting text against a hand-built, validated skill taxonomy 3. Reports what share of postings mention each skill — with the raw count, not just a percentage 4. Publishes it all through a live dashboard: single-role skill breakdowns, side-by-side role comparison with common/differentiator analysis, a market snapshot, skill search, and direct links to the real postings Every number distinguishes **observation** (what employers wrote) from **interpretation** — the dashboard's "About this data" section explains the methodology and limitations in full. ## How it fits together ### Data pipeline - **`scrapers/fetch_adzuna.py`** — pulls postings from the Adzuna API (South Africa). Primary skill-signal source — returns a real (if truncated to ~500 chars) description snippet. - **`scrapers/fetch_careers24.py`** / **`scrapers/fetch_fuzu.py`** — secondary sources, both honestly scoped to posting-volume tracking only (neither exposes enough description text for reliable skill matching — see their docstrings for why). - **`data/raw/manual_template.csv`** — same CSV schema, for sources without an API where postings are collected by hand. - **`skills_taxonomy.csv`** — flat synonym mapping (e.g. "Power BI" / "PowerBI" / "power-bi" → `power_bi`), spanning Technology, Finance, Healthcare, En …