APHRC-DSE/INSPIRE-VA-Data-to-OMOP-ETL

This repository contains code to ETL Verbal Autopsy data from various HDSS sites in Africa to an OMOP database instance # INSPIRE-VA-Data-to-OMOP-ETL This repository contains code to ETL Verbal Autopsy data from various HDSS sites in Africa to an OMOP database instance ## Requirements ### Hardware A laptop/desktop with the following specifications is recommended: - **Processor:** Intel Core i5 or higher (higher core count is beneficial for parallel processing) - **RAM:** 16GB (recommended for most OHDSI workloads). If you anticipate running multiple containers or handling large datasets, consider upgrading to 32GB. - **Storage:** Minimum 500GB, but more (1TB) is better with a Solid-State Drive (SSD) - **Operating System:** Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11. This allows you to install the Windows Subsystem for Linux (WSL) and WSL 2 using a single, streamlined command. ### Software Download and install - Java 1.8 or higher. But preferably Java JDK - White rabbit and Rabbit in a hat - Athena is a web-based resource tool for OMOP vocabulary. Link to Register and access - Usagi - DBMS(PostgreSQL) - **R software**, followed by **Rtools** corresponding to your R version then **RStudio/Posit IDE**. ## Data The data used for the ETL are available on reasonable request from the **Study PI - Agnes Kiragga** and **Study Program Manager - Ivan Busulwa**. - **Data used for ETL:** Data for Six Health and Demographic Surveillance System (HDSS) Sites in folders `VA_Hararghe_HDSS - Ethiopia`, `VA_IgangaMayuge_HDSS - Uganda`, `VA_MEIRU_Karonga_HDSS - Malawi`, `VA_Niakhar_HDSS - Senegal`, `VA_NUHDSS - Kenya` and `VA_Ouagadougou_HDSS - Burkina Faso` After permission is granted, download the data and save the six folders inside the data folder. Each individual folder contains Stata DTA files **(Do not rename the folders and DTA files)**. ## ETL Steps Followed 1. **Data processing:** Cleaning and Exploratory data analysis. 2. **Data Profiling:** Generate a Scan Report in White Rabbit using the CSV `VA_INSPIRE_merged_final.csv` generated in the data proces …