hashirama21/gik-icechain

Zero-Cost Cloud-Native Pipeline for Retrospective Flood Risk Decision Support in East Africa using the ECMWF IFS Ensemble # GIK-IceChain v2.0 **Zero-Cost Cloud-Native Pipeline for Retrospective Flood Risk Decision Support in East Africa using the ECMWF IFS Ensemble** > **ECMWF Code for Earth 2026 - Africa Stream (ArcX)** > Challenge 41: Missed Opportunities in Flood Disaster Risk Management > Mentors: Nishadh Kalladath · Masilin Gudoshava · Ahmed Amdihun · Anthony Mwanthi · Katherine Egan · Jessica Keune · Hillary Koros ## Overview ECMWF's open IFS ensemble archive on AWS S3 (`s3://ecmwf-forecasts`) holds over **1 petabyte** of GRIB2 weather forecast data: 51 ensemble members, 85 forecast steps (0-360 h), and ~1 250 forecast days since 18 January 2023 - all freely available. Yet this data is almost completely inaccessible to the xarray/Dask ecosystem that East African disaster risk managers rely on. GIK-IceChain v2.0 solves this in three components: | Component | What it does | Key output | |-----------|-------------|-----------| | **C1 - Conversion** | GIK Parquet manifests → VirtualiZarr → IceChunk Zarr v3 virtual store | Single `zarr.open()` access to 1 200+ days, zero data duplication | | **C2 - Exceedance** | Retrospective rainfall exceedance probabilities for all ~1 200 days | Multi-dim Zarr: `date × lat × lon × window × return_period` | | **C3 - Risk (CRMA)** | Admin-1 daily flood risk using ICPAC's CRMA Bayesian Network | Daily GeoJSON risk layer, integrated in calendar-map storymaps | ### Key numbers | Metric | Value | |--------|-------| | Archive size | ~1 PB (raw GRIB2) | | Virtual store size | ~18.5 GB (metadata only) ≈13 000× smaller than an equivalent full Zarr copy | | Days covered | 1 246 (2023-01-18 → present, verified 2026-07 against the published store) | | Ensemble members | 51 | | Accumulation windows | 7 (3 h → 7 days) | | Return-period thresholds | 6 (2, 5, 10, 20, 40, 100 years) | | Admin-1 units | 238 (16 East African countries) | ## Architecture Pipeline diagram — horizontal flow, scroll sideways ```mermaid flowchart LR %% ================ …