HossamJa/Historical-Weather-Forecast-Comparison-to-Actuals
Domain:
climateCreator:
Hos
Host:
This project is a proof-of-concept (POC) ETL (Extract, Transform, Load) pipeline that automates the collection of daily weather data for Casablanca, Morocco.
# Weather ETL Automation – Casablanca Forecast Accuracy Tracker
## Project Overview
This project is a proof-of-concept (POC) ETL (Extract, Transform, Load) pipeline that automates the collection of daily weather data for Casablanca, Morocco. It extracts the observed temperature and the forecasted temperature for the next day, transforms the data into a structured format, and logs it in a tabular format. This data will be used by the analytics team to evaluate the historical accuracy of weather forecasts.
The project was completed as part of an IBM course focused on Linux commands and shell scripting.
## Learning Objectives
Through this project, I gained practical experience in:
- Using `curl` to retrieve data from web APIs
- Parsing and extracting meaningful information from text output
- Transforming raw data into a tabular format
- Automating processes using shell scripts and `cron` jobs
- Writing readable, maintainable, and modular shell scripts
## Data Source
Weather data is sourced from wttr.in, a free and open-source weather service that delivers forecasts in a terminal-friendly format.
Example data retrieval:
```bash
curl wttr.in/casablanca
```
## Extracted Data
The pipeline captures the following fields every day at noon (local time):
| Year | Month | Day | Observed Temp (°C) | Forecasted Temp (°C) |
| ---- | ----- | --- | ------------------ | -------------------- |
| 2023 | 1 | 1 | 10 | 11 |
| 2023 | 1 | 2 | 11 | 12 |
| 2023 | 1 | 3 | 12 | 10 |
The log is saved in a file (weather_report.csv or similar), where each row corresponds to a new day.
## Project Structure
```bash
weather-etl/
├── weather_etl.sh # Main shell script to extract, transform, and log the data
├── weather_report.csv # Output file storing daily temperature data
└── README.md # Project documentation
```
## Automation
The process …