abrhagebrehiwet/ethiopia-job-market-eda

Web Scraping and Exploratory Data Analysis of Job Market Trends in Ethiopia (SDG 8) is a data science project that scrapes job vacancy data from etcareers.com and analyzes Ethiopia’s labor market trends. The project uses Python tools such as pandas, numpy, matplotlib, and regex processing to clean job postings, extract important information, and vi # Web Scraping and EDA of Job Market Trends in Ethiopia This project supports **Sustainable Development Goal 8: Decent Work and Economic Growth** by collecting and analyzing job vacancy data from ETCareers. It combines web scraping, data cleaning, regex-based text processing, exploratory data analysis, and visualization to study job-market trends in Ethiopia. ## Project Objectives - Scrape job listings from `etcareers.com` into a structured CSV dataset. - Extract key fields such as job title, company, location, deadline, job type, and posting URL. - Clean and standardize scraped data using `pandas`, `numpy`, and regex processing. - Analyze sector trends, location distribution, experience demand, and skill frequency. - Generate clear visualizations using `matplotlib` and project-ready summary tables. - Keep the workflow reproducible so the scraper can be rerun as new postings appear. ## Repository Structure ```text ethiopia-job-market-eda/ |-- data/ | |-- raw/ # Original scraped or manually collected job-posting data | `-- processed/ # Cleaned analysis-ready datasets |-- docs/ | |-- data_dictionary.md # Expected columns and definitions | `-- methodology.md # Analysis workflow and assumptions |-- notebooks/ # Exploratory notebooks |-- reports/ | `-- figures/ # Generated charts |-- src/ | `-- ethiopia_job_market_eda/ | |-- clean.py # Cleaning pipeline | |-- cli.py # Scrape and analysis command-line runner | |-- config.py # Project paths and schema | |-- eda.py # Summary-table helpers | |-- scraper.py # Web scraper helpers | |-- text_processing.py # Regex skill and experience extraction | `-- visualization.py # Chart generation |-- requirements.txt `-- pyproject.toml ``` ## Scraping Workflow Run the scraper against the ETCareers jobs page: ```powershell python -m ethiopia_job_market_eda.cli scrape "etc …