Pivot-Madagascar/smaller-backend-clean

a lighter version of smaller-backend # SMALLER-backend ## Description This repo contains the backend pipeline for the SMALLER malaria prediction dashboard. It is run monthly to update the environmental and health data and rerun the predictive models to result in predictions for the next two months. Predictions are then visualized via the SMALLER dashboard. ## System Requirements - R v. 4.3.1 or above - INLA v. 23.04.24 (installation details - python v 3.12+ - Google Earth Engine service account(details) - internet connection of at least 10 Mbps to download ~ 100MB of data monthly. Note that the SEAS-IO download will time out if download speed is too slow. - 16 GB RAM - 100 MB of stockage per year of data (will depend on variables and number of spatial zones included) The following R packages are required: ``` [1] "tidyr" "lubridate" "sf" "dplyr" "stringr" "ggplot2" "parallel" [8] "spdep" "dlnm" "tsModel" "purrr" "hoardr" "INLA" "INLAutils" [15] "inlatools" "zoo" "skimr" "knitr" "quarto" ``` The following python libraries are required: ``` ee earthengine-api pandas geopandas dotenv pridec_gee @ git+github.com ``` The pipeline is currently run with the following specifications: ``` R version 4.3.1 (2023-06-16) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 22.04.3 LTS Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so; LAPACK version 3.10.0 ``` The full workflow, including one month of downloading and updating data, currently takes 5 minutes when run on 6 cores using a Intel® Core™ i7-10850H CPU with an internet connection of 20 Mbps. ## Installation Guide This repo can be installed via cloning. It then contains all of the scripts to be run in a `targets` pipieline in the `_targets.R` file. Two additional SECRETS files are required to run the targets workflow. They sh …