mrrobotke/iJobs-Scraper

Job scraping & AI enrichment engine for African job markets # ijobs-scraper Job scraping & AI enrichment engine for African job markets. ## Features - **13 built-in adapters** covering Kenyan job portals — API, HTML, and browser-based sources - **AI enrichment** via OpenAI structured outputs — extracts titles, skills, salary, categories, and more from raw listings - **3-layer deduplication** — source URL uniqueness, SHA-256 content hashing, and optional fuzzy matching - **Async-first** — all I/O uses `async`/`await` with `httpx` for HTTP and `playwright` for browser automation - **Framework-agnostic** — zero coupling to FastAPI, Django, or any web framework. Integrate anywhere via Protocol-based dependency injection - **Type-safe** — Pydantic v2 models throughout, passes `mypy --strict` - **Cron scheduling** — built-in schedule evaluation via `croniter`. Your app decides when and how to enqueue - **Open-source** — MIT licensed, designed for contributors to add new portal adapters in ~50 lines of code ## Quick Start ```bash pip install ijobs-scraper ``` ```python import asyncio from ijobs_scraper import ScraperEngine, SourceConfig, SourceType class StubAIProvider: """Minimal AI provider for testing — returns mock enriched data.""" async def structured_extract(self, system_prompt, user_prompt, json_schema): return { "title": "Software Engineer", "description": "A great role.", "company_name": "One Acre Fund", "company_website": None, "location": "Nairobi, Kenya", "remote_type": "hybrid", "employment_type": "full_time", "experience_level": "Mid-level", "salary_min": None, "salary_max": None, "currency": "KES", "skills": ["Python", "SQL"], "benefits": [], "category": "technology-engineering", "requirements": None, "posted_at": None, "expires_at": None, } async def main(): engine = ScraperEngine(ai_provider=StubAIProvider()) source = SourceConfig( name="One Acre Fund", slug="one-acre-fund", adapter="greenhouse", source_type=SourceType.API, base_url="boards-api.greenhouse.io", config={"board_token": "oneac …