abrhagebrehiwet/ethiopia-job-market-eda
Domain:
socioeconomicCreator:
abr
Host:
Web Scraping and Exploratory Data Analysis of Job Market Trends in Ethiopia (SDG 8) is a data science project that scrapes job vacancy data from etcareers.com and analyzes Ethiopia’s labor market trends. The project uses Python tools such as pandas, numpy, matplotlib, and regex processing to clean job postings, extract important information, and vi
# Web Scraping and EDA of Job Market Trends in Ethiopia
This project supports **Sustainable Development Goal 8: Decent Work and Economic Growth** by collecting and analyzing job vacancy data from ETCareers. It combines web scraping, data cleaning, regex-based text processing, exploratory data analysis, and visualization to study job-market trends in Ethiopia.
## Project Objectives
- Scrape job listings from `etcareers.com` into a structured CSV dataset.
- Extract key fields such as job title, company, location, deadline, job type, and posting URL.
- Clean and standardize scraped data using `pandas`, `numpy`, and regex processing.
- Analyze sector trends, location distribution, experience demand, and skill frequency.
- Generate clear visualizations using `matplotlib` and project-ready summary tables.
- Keep the workflow reproducible so the scraper can be rerun as new postings appear.
## Repository Structure
```text
ethiopia-job-market-eda/
|-- data/
| |-- raw/ # Original scraped or manually collected job-posting data
| `-- processed/ # Cleaned analysis-ready datasets
|-- docs/
| |-- data_dictionary.md # Expected columns and definitions
| `-- methodology.md # Analysis workflow and assumptions
|-- notebooks/ # Exploratory notebooks
|-- reports/
| `-- figures/ # Generated charts
|-- src/
| `-- ethiopia_job_market_eda/
| |-- clean.py # Cleaning pipeline
| |-- cli.py # Scrape and analysis command-line runner
| |-- config.py # Project paths and schema
| |-- eda.py # Summary-table helpers
| |-- scraper.py # Web scraper helpers
| |-- text_processing.py # Regex skill and experience extraction
| `-- visualization.py # Chart generation
|-- requirements.txt
`-- pyproject.toml
```
## Scraping Workflow
Run the scraper against the ETCareers jobs page:
```powershell
python -m ethiopia_job_market_eda.cli scrape "etc …