mouhamedhabib/Tunisia-Jobs-Tracker
Domain:
socioeconomicRecord type:
software
Creator:
mou
Host:
A Python web scraping pipeline that collects job listings from major Tunisian job boards, stores them in a structured database, and visualizes the data through an interactive dashboard.
# 🇹🇳 Tunisia Jobs Tracker
A Python web scraping pipeline that collects job listings from major Tunisian job boards, stores them in a structured database, and visualizes the data through an interactive dashboard.
🔗 **Live Demo:** tn-jobs-tracker.streamlit.app
---
## 📊 Dashboard Preview
> Built with Streamlit — filters by city, contract type, and source.
---
## 🗂️ Data Sources
| Source | Pages | Status |
|--------|-------|--------|
| farojob.net | 782 | ✅ Active |
| keejob.com | ~100 | ✅ Active |
| offre-emploi.tn | 43 | ✅ Active |
---
## 🏗️ Project Structure
```
tunisia-jobs-tracker/
├── scrapers/
│ ├── base_scraper.py # Shared base class (session, rate limiting, logging)
│ ├── farojob.py # Scraper for farojob.net
│ ├── keejob.py # Scraper for keejob.com
│ └── offre_emploi_tn.py # Scraper for offre-emploi.tn
├── processing/
│ └── cleaner.py # Data cleaning + city normalization + CSV export
├── database/
│ └── db.py # SQLite + migration system
├── dashboard/
│ └── app.py # Streamlit interactive dashboard
├── data/
│ └── jobs_cleaned.csv # Latest scraped data (auto-updated daily)
├── .github/workflows/
│ └── scrape.yml # GitHub Actions — runs scrapers daily at 6AM UTC
├── .env.example # Environment template
└── requirements.txt
```
---
## ⚙️ Tech Stack
- **Python 3.12**
- **requests + BeautifulSoup4** — HTML parsing
- **SQLite** — lightweight local database with migration system
- **pandas** — data cleaning and normalization
- **Streamlit** — interactive dashboard
- **GitHub Actions** — daily automated scraping
- **python-dotenv** — environment configuration
---
## 🚀 Getting Started
> **Windows users:** This project was developed and tested on **WSL (Windows Subsystem for Linux)** with Ubuntu. It is strongly recommended to run it inside WSL rather than native Windows to avoid path and encoding issues.
>
> To enable WSL: `wsl …