pmwaniki/retro-code

Code and datasets: Retrospective Evaluation of a Generative AI-Enabled Electronic Medical Record System in Primary Health Care Facilities in Kenya ## setup Create a file named ".env" in the root directory for specifying environmental variables. It should containe the environmental variables variables "DATA_FOLDER" (location of datasets) and ""OUTPUT_FOLDER" (where outputs will be saved). The following dataset are required to run the scripts: - *hashed_raw_data.parquet* - The raw data with columns having identifying information hashed - *hashed_visit_info.parquet* - Contains gender, age, chief complaint, and diagnoses of patients in "hashed_raw_data.parquet" - *ai_consult_freq.csv* - Aggregated data on LLM usage during the study period - *hashed_recoded_diagnoses.json* - A dictionary that maps "diagnoses" in "hashed_visit_info.parquet" into affected body system, eg Gastrointestinal ## Inter-rater reliability This analysis assesses the consistency among multiple raters' evaluations. The `agreement.R` script uses the IRR package in R to calculate Kendall's W for ordinal variables and Fleiss' kappa for nominal variables, quantifying inter-rater reliability. These metrics indicate the degree of agreement and help validate the objectivity and quality of the rating process. ## Descriptive analyses This section presents descriptive statistics for categorical variables in the dataset, including frequency counts and proportions for each category. The `descriptives.R` script summarizes the distribution of categorical data, highlighting the prevalence of different categories and enabling a clear understanding of the dataset's composition. These analyses provide essential context for interpreting subsequent statistical results.