ikay7025/Africa-tech-jobs-scraper

Modular Selenium-based job scraping pipeline # Africa Tech Jobs Scraper ## Overview A modular web scraping pipeline built with Python and Selenium for extracting structured job listings from dynamic web pages. The project demonstrates end-to-end automation including data extraction, cleaning, logging, and structured export. --- ## Key Features - Dynamic web scraping using Selenium - Modular architecture (scraper, cleaner, exporter) - Data validation and deduplication - Config-driven setup for flexibility - Logging system for debugging and monitoring - Export to CSV and JSON formats --- ## System Architecture Scraping Pipeline: Scraper (Selenium) → Raw Data Extraction → Cleaner (Validation & Deduplication) → Exporter (CSV / JSON Output) → Output Files --- ## Tech Stack - Python - Selenium - Pandas - WebDriver Manager --- ## Project Structure africa-tech-jobs-scraper/ │ ├── scraper.py ├── cleaner.py ├── exporter.py ├── logger.py ├── config.py ├── requirements.txt │ ├── output/ │ ├── jobs.csv │ ├── jobs.json │ └── logs/ └── scraper.log ## How to Run ```bash pip install -r requirements.txt python scraper.py --- Output The pipeline generates structured datasets: jobs.csv jobs.json Located in the /output directory ------- Use Cases -Job market data extraction -Recruitment intelligence systems -AI training dataset generation -Automation and data pipeline demonstrations ------- Author Ikechukwu Emmanuel chrisikechukwu22@gmnail.com