ssmurfgg04-gif/jiji-deal-hunter

Multi-market classifieds intelligence platform for Jiji (Kenya, Nigeria, Ghana, Tanzania, Uganda). API-first collector with recon-derived scam signals, image-hash dedup, XGBoost deal scoring, and real-time dashboard. # Jiji Deal Hunter Multi-market classifieds intelligence platform for Jiji.co.ke (and Nigeria, Ghana, Tanzania, Uganda). API-first collector with recon-derived scam signals, image-hash dedup, XGBoost deal scoring, and a real-time dashboard. ## ⚠️ Read this first **Cloudflare blocks datacenter IPs.** Jiji.co.ke sits behind Cloudflare's anti-bot challenge. From a cloud/datacenter IP, the API returns 403 + a JS challenge page. From a residential IP, it returns clean JSON. This is **not** something httpx, curl, or any plain HTTP client can bypass — Cloudflare's challenge requires executing JavaScript in a real browser context. Libraries that spoof TLS fingerprints (curl_cffi etc.) exist specifically to circumvent this, and I won't use them. **What this means for you:** - The dashboard's `BLOCKED` badge will show red when run from a cloud server. - Deploy to a VPS on a residential ISP, or run locally on your home connection — the badge flips to `LIVE API` automatically. - The Wayback miner (`scripts/wayback-miner.ts`) works from anywhere because it queries `web.archive.org`, not Jiji directly. ## 5-minute setup ```bash git clone github.com cd jiji-deal-hunter bun install # 1. Push schema to SQLite + apply WAL mode + indexes bun run db:push # 2. Seed 16 verified real archived listings (from recon) bun scripts/seed-csv.ts # 3. Mine ~800 more real archived listings from web.archive.org # (requires internet access to web.archive.org — runs from your machine) bun scripts/wayback-miner.ts # 4. Train XGBoost on real archived data python3 scripts/train-xgboost.py # 5. Score all listings bun scripts/rescore.ts # 6. Start the dashboard bun run dev ``` Open `localhost`. The dashboard auto-seeds on first load. ## What it does - **Live API collection** — Direct calls to `api_web/v1/*` endpoints (no browser, no Cloudflare bypass). Multi-market support. - **Wayback mining** — Passive archival mining of ~800 hist …