M3nnoun/rap-morocco

# 🎤 Moroccan Rap — Dataset & Collaboration Map A research project mapping the Moroccan rap scene from Genius: a reproducible scraping pipeline that builds a lyrics-ready dataset, plus an **interactive collaboration map** of who works with whom. **🌐 Live map:** m3nnoun.github.io > This is the **data-collection + network-analysis** phase of a larger NLP project. > The planned NLP work (topic modeling per artist, similar-artist and > similar-track discovery) builds on the dataset produced here. --- ## The collaboration map Every node is a rapper; a link means they appear together on a track (a shared song on Genius). Node size = number of collaborators, color = detected community ("scene"). The layout is precomputed and frozen for smooth, jitter-free exploration. - **283 rappers · 844 collaborations · 16 scenes** - Click a rapper to spotlight their collaborators, search by name, toggle labels and solo artists. The map is a single self-contained `index.html` (the vis-network library is inlined) — it works offline and is what GitHub Pages serves as the site. Regenerate it from the data with: ```bash python -m scraper.build_collab_map ``` --- ## The dataset | | | |---|---| | Artists | **283** | | Songs (metadata) | **10,754** (own + features) | | Source | genius.com | | Format | one JSON file per artist in `data/artists/` | Built in three resumable stages. Genius sits behind a Cloudflare interactive challenge that blocks plain HTTP clients (`requests`/`httpx` get `403`) and even vanilla Playwright, so the pipeline uses **patchright** (a stealth Playwright fork) with a persistent, headed browser profile to clear it. | Stage | Module | Output | |------|--------|--------| | 1. Discover artists | `scraper.stage1_artists` | `data/artists_index.json` (601 names from the editorial list) | | 2. Discover songs | `scraper.stage2_songs` | per-artist files with song stubs | | 3. Scrape lyrics | `scraper.stage3_lyrics` | raw lyrics filled into the p …