Lesotho-Meteorological-Services/geodata-auto-ingest

An utility to pull geoTIFF/neCDF files from remote sources for ingesting into Climweb/Geomanager # GEODATA-AUTO-INGEST A repo-friendly automation package for preparing and auto-ingesting geospatial datasets into GeoManager/Climweb. This repo now supports both major raster workflows cleanly: - **GeoTIFF workflow** - CHIRPS v3 Africa dekadal rainfall → Lesotho clip → GeoManager auto-ingest - LMS or other local GeoTIFF sources → optional clip → GeoManager auto-ingest - **NetCDF workflow** - Local or remote `.nc` files → optional variable validation → GeoManager auto-ingest The structure is meant to grow. You can add more source-specific ingestors later, for example LMS forecast rasters, NDVI/VHI rasters, JRC WSI rasters, or sector-specific NetCDF products. --- ## Repo layout ```text GEODATA-AUTO-INGEST/ ├── pyproject.toml ├── README.md ├── env/ │ ├── chirps_v3_africa.env.example │ ├── lms_geotiff_template.env.example │ └── netcdf_template.env.example ├── scripts/ │ ├── run_chirps_v3_africa.sh │ ├── run_lms_geotiff_template.sh │ └── run_netcdf_template.sh └── src/ └── geodata_auto_ingest/ ├── __init__.py ├── common/ │ ├── __init__.py │ ├── chirps.py │ ├── ingest.py │ ├── logging_utils.py │ └── shell.py └── sources/ ├── __init__.py ├── chirps_v3_africa.py ├── lms_geotiff_template.py └── netcdf_template.py ``` --- ## What GeoManager expects GeoManager treats GeoTIFF and NetCDF differently. ### GeoTIFF GeoManager auto-ingest for GeoTIFF depends on the filename ending with an ISO UTC timestamp in this form: ```text YYYY-MM-DDTHH:MM:SS.sssZ ``` Example: ```text chirps-v3.0.2026.02.3-LSO_2026-02-21T00:00:00.000Z.tif ``` That is why the GeoTIFF ingestors in this repo create staged filenames with that suffix. ### NetCDF GeoManager supports `.nc` files and extracts time from the file itself. The layer in GeoManager must have: - **Auto ingest from directory** enabled - **Data variable for netCDF data auto ingest** set to the correct variable name This repo does not rename NetCDF files with timestamps because GeoManager reads time from …