arinacraciun/Africa-Weather

## Physics-Informed Graph Neural Networks for Atmospheric Forecasting This repository contains a complete, end-to-end machine learning pipeline for processing massive meteorological datasets, training a Graph Neural Network (GNN) to simulate atmospheric dynamics, and evaluating autoregressive weather forecasts. The system is designed to predict multi-variable weather states over the African continent using ERA5 reanalysis data and ground-truth station observations. ### **Project Overview** Modern data-driven weather models must overcome significant challenges: planetary-scale data volume, polar distortion on standard grids, and exponential error accumulation during multi-step forecasts. This project addresses these via an out-of-core data pipeline, a spherical graph topology, and a physics-informed residual modeling approach. ### **Pipeline Architecture** | Component | File | Responsibilities | Core Technologies | | --- | --- | --- | --- | | **Data Engineering** | `data_preparation.py` | ERA5 downloading, lazy loading, Zarr chunking, and spherical graph generation. | `cdsapi`, `xarray`, `dask`, `scikit-learn` | | **Model Training** | `model_training.py` | PyG data collation, physics-informed loss computation, and mixed-precision training. | `pytorch`, `pytorch_lightning`, `torch_geometric` | | **Evaluation** | `model_evaluation.py` | Autoregressive inference, graph-to-grid reconstruction, metric calculation, and visualization. | `xskillscore`, `cartopy`, `matplotlib` | --- ### **Core Technical Achievements** * **Massive Data Handling (Out-of-Core Processing):** Implemented lazy loading and optimized chunking using `xarray` and `dask` to process high-resolution, multi-dimensional ERA5 GRIB files without exhausting system memory. Data is serialized into a flattened, Anemoi-compatible Zarr format for high-throughput model ingestion. * **Spherical Graph Construction:** Solved the "pole problem" inherent in 2D latitude/longitude grids by mapping coordinates to 3D …