WorldFishCenter/peskas.kenya.data.pipeline

# peskas.kenya.data.pipeline The goal of peskas.kenya.data.pipeline is to implement, deploy, and execute the data and modelling pipelines that underpin Peskas in Kenya, a partnership between WorldFish and Wildlife Conservation Society as part of the Asia-Africa Bluetech Superhighway project funded by the FCDO of the UK Government. ## The pipeline is an R package peskas.kenya.data.pipeline is structured as an R package because it makes it easier to write production-grade software. Specifically, structuring the code as an R package allows us to: - better handle system and package dependencies, - forces us to split the code into functions, - makes it easier to document the code, and - makes it easier to test the code We make heavy use of tidyverse style conventions and the usethis package to automate tasks during project setup and deployment. For more information about the rationale of structuring the pipeline as a package check Chapter 3 in *Engineering Production-Grade Shiny Apps*. The book is focused on Shiny applications but the rationale also applies to data pipelines and production-ready code in general. ## How the pipeline works The pipeline is composed of different modules: 1. Data Collection: On site fishing landing surveys and continuous, solar-powered GPS vessel trackers to collect and send data in near real-time, alongside fishery metadata for a thorough data-gathering process. 2. Pre-processing: Data formatting, shaping, and standardisation to prepare the raw data for analysis. 3. Validation: Outlier detection and error identification, and includes an alert system to maintain data quality. 4. Analytics: Modelling fisheries indicators, nutritional characterization, and data mining to extract valuable insights. 5. Data export: Automated dissemination of processed and analysed fisheries data to ensure accessibility and comprehension. This involves restructuring data for dashboard integration and open publication. 6. isualisation: Too …