itshopedev25-rgb/african-music-platform
Record type:
software
Creator:
its
Host:
African Music Intellegence
# African Music Intelligence Platform
A data pipeline that tracks Ghanaian music artists across genres (Asakaa/Drill, Afro-fusion,
Hiplife, Dancehall, Afropop, Afro-dancehall) using the Spotify Web API, and surfaces their
release activity through a dashboard.
**Status: personal/portfolio project**, not a commercial product. See Scope & Spotify
Terms before reusing this for anything beyond learning.
## Why a seed list instead of crawling playlists?
Spotify's Web API has tightened significantly for apps in Development Mode (the tier this
project runs under):
- Reading the track contents of playlists owned by other users (`playlist_items`) is blocked.
- `followers`, `popularity`, `genres`, `top-tracks`, and `related-artists` are no longer
returned at all for this app tier, even with a logged-in user token.
- Playlist search (`search(type="playlist")`) requires the app-owning account to have an
active Spotify Premium subscription.
Because of this, the pipeline starts from a manually curated list of known artists
(`SEED_ARTISTS` in `src/scraper.py`) and pulls each one's own release history directly,
rather than discovering artists by crawling genre playlists.
## Pipeline
```
scraper.py Search Spotify for each seed artist, fetch their albums/singles,
upload incrementally to S3 (raw/spotify/artists.json)
|
v
quality_check.py Validate the raw data: shape, nulls, duplicates, dtypes
|
v
cleaner.py Flatten to one row per release, fix release_date (handles
year-/month-only precision), write Parquet to S3:
- processed/spotify/releases.parquet (latest, overwritten)
- processed/spotify/dt= /releases.parquet (daily snapshot,
auto-expires after 30 days via an S3 lifecycle rule)
|
v
metrics.py Aggregated tables: total releases per artist, album-vs-single
trends.py breakdown, releases per year, most active artists, album/single
insights.py share, monthly trends, rolling averages, plain-English insights
|
v
app.py …