mlarh-16/us_per_states_africa_trade
Domain:
socioeconomicRecord type:
dataset
Creator:
mla
Host:
US state-level import and export trade data with African countries, sourced from the US Census Bureau International Trade API (2020–2025).
# US State × Africa Trade Data
A Python pipeline that scrapes US state-level import and export trade data with African countries from the US Census Bureau International Trade API, covering all 50 states, DC, and US territories against 60 African nations from 2020–2025.
## Features
- Fetches data from the Census Bureau International Trade API across all state × flow × year combinations
- Filters to the official Census Africa country grouping (~60 nations)
- Handles API name aliases and normalizes country names (e.g. "Cape Verde" → "Cabo Verde", "Congo (Kinshasa)" → "Democratic Republic of the Congo")
- HS2 commodity classification mapped to 5 custom trade categories (Foods & Beverages, Industrial Supplies, Consumer Goods, Capital Goods, Automotive)
- Retry logic with exponential backoff for resilient API fetching
- Data integrity validation: duplicate key checks and negative value detection
- Outputs both CSVs and a multi-sheet Excel workbook
- CLI with `--test`, `--year`, and `--flow` flags
## Requirements
- Python 3.11+
- A Census Bureau API key — free registration at api.census.gov
## Setup
```bash
# Clone the repository
git clone github.com
cd fifa-us-africa-trade
# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
# Install dependencies
pip install -e .
# Set your Census API key
export CENSUS_API_KEY=your_key_here # Windows: set CENSUS_API_KEY=your_key_here
```
## Usage
All commands should be run from the `scripts/` directory.
```bash
cd scripts
# Quick 3-state test (TX, NJ, GA × both flows × 2023)
python tradestats_scraper.py --test
# Full scrape: all states × both flows × 2020–2025
python tradestats_scraper.py
# Single year
python tradestats_scraper.py --year 2024
# Exports only
python tradestats_scraper.py --flow export
# Imports only, single year
python tradestats_scraper.py --flow import --year 2023
```
## Outpu …