itshopedev25-rgb/african-music-platform

African Music Intellegence # African Music Intelligence Platform A data pipeline that tracks Ghanaian music artists across genres (Asakaa/Drill, Afro-fusion, Hiplife, Dancehall, Afropop, Afro-dancehall) using the Spotify Web API, and surfaces their release activity through a dashboard. **Status: personal/portfolio project**, not a commercial product. See Scope & Spotify Terms before reusing this for anything beyond learning. ## Why a seed list instead of crawling playlists? Spotify's Web API has tightened significantly for apps in Development Mode (the tier this project runs under): - Reading the track contents of playlists owned by other users (`playlist_items`) is blocked. - `followers`, `popularity`, `genres`, `top-tracks`, and `related-artists` are no longer returned at all for this app tier, even with a logged-in user token. - Playlist search (`search(type="playlist")`) requires the app-owning account to have an active Spotify Premium subscription. Because of this, the pipeline starts from a manually curated list of known artists (`SEED_ARTISTS` in `src/scraper.py`) and pulls each one's own release history directly, rather than discovering artists by crawling genre playlists. ## Pipeline ``` scraper.py Search Spotify for each seed artist, fetch their albums/singles, upload incrementally to S3 (raw/spotify/artists.json) | v quality_check.py Validate the raw data: shape, nulls, duplicates, dtypes | v cleaner.py Flatten to one row per release, fix release_date (handles year-/month-only precision), write Parquet to S3: - processed/spotify/releases.parquet (latest, overwritten) - processed/spotify/dt= /releases.parquet (daily snapshot, auto-expires after 30 days via an S3 lifecycle rule) | v metrics.py Aggregated tables: total releases per artist, album-vs-single trends.py breakdown, releases per year, most active artists, album/single insights.py share, monthly trends, rolling averages, plain-English insights | v app.py …