farrakhanwahab/jsg-etl

An ETL project for The Judicial Service of Ghana – Data Warehousing & Mining # Complete Judicial Service ETL Pipeline & Analytics Dashboard A comprehensive Extract, Transform, Load (ETL) solution with modern analytics dashboard for the Judicial Service of Ghana. This project provides a complete data warehousing and business intelligence solution. ## What This Project Provides ### Complete ETL Pipeline - Extract: Automated data extraction from multiple sources (CSV, Excel, JSON) - Transform: Data cleaning, standardization, and business logic application - Load: Automated loading into MySQL data warehouse - Quality: Data quality validation and reporting ### Modern Analytics Dashboard - Beautiful UI: Modern, responsive design with dark/light themes - Interactive Charts: Real-time visualizations using Chart.js - Real-time Data: Live data from your MySQL data warehouse - Multiple Views: Overview, Cases, Courts, Judges, and Trends ## Quick Start (Complete Pipeline) ### Prerequisites - Python 3.9+ - XAMPP (MySQL running) - Judicial data files in `Data/`, `mock_data/`, or `generated_data/` folders ### One-Command Setup 1. Start XAMPP MySQL and ensure it's running 2. Run the complete pipeline: ```bash ./run_complete_pipeline.sh ``` 3. Open your browser: ``` localhost ``` This single command will: - Install all dependencies - Run the complete ETL pipeline - Perform data quality checks - Start the modern dashboard ## Individual Components ### ETL Pipeline Only ```bash python3 etl_pipeline.py ``` ### Data Quality Check Only ```bash python3 data_quality_checker.py ``` ### Dashboard Only ```bash cd modern_dashboard ./start_dashboard.sh ``` ## ETL Pipeline Features ### Data Extraction - Multiple Formats: CSV, Excel, JSON support - Multiple Sources: Data/, mock_data/, generated_data/ folders - Error Handling: Robust error handling and logging - Flexible: Easy to add new data sources ### Data Transformation - Cleaning: Handle missing values, duplicates, format standardization - Business Logic: Calculate resolution times, vali …