05Menna/Egypt-Weather-Analysis-using-MapReduce

Egypt Weather analysis using MapReduce # Weather Analysis with Map-Reduce ## 📋 Project Overview This project analyzes a comprehensive **weather dataset covering all Egyptian governorates from 2000 to 2024**. The primary focus is transforming raw weather logs into meaningful insights through data cleaning, preprocessing, feature engineering, and visualization using **MapReduce** for distributed data processing. The project demonstrates advanced data engineering techniques including: - Data acquisition and preprocessing - Statistical analysis and outlier detection - Feature engineering and distribution analysis - MapReduce implementation for scalable computation - Interactive visualizations and climate trend analysis --- ## 👥 Team Members - **Menna Mohammed Abdelbaky** - **Mariam Salah Abdul Muttalib El-meligy** - **Sondos Yasser Mohamed Fahmy Hashem** - **Sohier Mohamed Salah Eldin Mohamed** - **Manar Samir Ali Diab** --- ## 📊 Dataset Description ### Data Source Meteostat Weather and Climate Database ### Dataset Coverage - **Time Period:** 2000–2024 (25 years) - **Geographic Scope:** All 23 Egyptian governorates - **Daily Observations:** Weather measurements recorded daily ### Dataset Columns | Column | Description | Unit | |--------|-------------|------| | **date** | Date of observation (derived from year, month, day) | — | | **city** | Egyptian governorate name | — | | **year** | Year of observation | 2000–2024 | | **month** | Month of the year | 1–12 | | **day** | Day of the month | 1–31 | | **pressure** | Atmospheric pressure | hPa | | **maxtemp** | Maximum daily temperature | °C | | **temperature** | Average daily temperature | °C | | **mintemp** | Minimum daily temperature | °C | | **rainfall** | Total daily rainfall | mm | | **sunshine** | Sunshine duration | hours | | **windspeed** | Wind speed | km/h | | **week** | ISO week number (for imputation) | 1–53 | --- ## 🔄 Project Workflow ### 1. **Data Scraping** - Fetched real-time weather data from Meteostat API - Identified nearest wea …