Usability, Functionality, and User Competencies of the Electronic Case-Based Disease Surveillance (eCBDS) System in Sierra Leone: A Multi-Level Mixed-Methods Evaluation (Preprint)

BACKGROUND Abstract Background. Sierra Leone deployed the electronic Case-Based Disease Surveillance (eCBDS) system on the DHIS2 platform to move priority-disease notification from paper-based registers to real-time individual-level digital reporting. Whether frontline users across the health system possess the competencies the platform assumes, and whether the system is experienced as usable across a heterogeneous and infrastructurally uneven health system, had not been evaluated since national rollout. OBJECTIVE The objective of this study was to assess the usability, functionality and user competencies associated with the eCBDS platform across the facility, district, laboratory and national levels of the Sierra Leone health system. The unit of analysis is the system and its users—not the surveillance data they produce, and not the infrastructure on which they depend. Six specific sub-objectives operationalised this aim: 1. To characterise the coverage and modality of eCBDS and IDSR training across facility, district and laboratory levels, and to document the reasons given by untrained personnel. 2. To determine which specific eCBDS competencies were transmitted at each level of the health system, and to identify where competency provision diverges from role expectations. 3. To assess objective user competency by grading respondents’ procedural understanding of the eCBDS workflow against a three-tier rubric. 4. To evaluate perceived system attributes—ease of use, ease of navigation, ease of learning, flexibility, acceptability, device compatibility and analytic self-efficacy—at facility and district levels. 5. To test whether perceived usability is equitably distributed across the four regions of the country. 6. To explain the quantitative usability findings through thematic analysis of user narratives. METHODS Methods. A concurrent embedded mixed-methods evaluation was conducted between January and July 2022 in eight of Sierra Leone’s sixteen districts, covering 40 health facilities, eight district health management teams and five regional referral laboratories (n = 56 questionnaire respondents), together with 16 analysable key informant interviews at national and district levels. Training coverage, competency domains and Likert-scaled system attributes were analysed descriptively. Procedural knowledge of the eCBDS workflow was graded against a three-tier rubric. Regional differences in attribute ratings were tested by one-way analysis of variance with Tukey’s HSD post-hoc comparison. Interviews were transcribed verbatim and thematically coded in NVivo. RESULTS Results. eCBDS training reached 35 of 40 (88%) facility respondents, 10 of 10 (100%) district respondents and three of five laboratory respondents. In-class delivery predominated (facility 87%, district 90%), and 51% of facility respondents additionally received one-to-one facilitation. Android data entry was near-universal (facility 85%, district 90%), but advanced competencies clustered at district level: web data entry 80% versus 33%, technical troubleshooting 50% versus 15% and device management 40% versus 21%. Only 19 of 50 (38%) combined facility and district respondents demonstrated adequate procedural knowledge of the eCBDS workflow, while 12 (24%) were graded inadequate. Ease of use, ease of navigation and ease of learning were endorsed by a majority of facility respondents, yet a comparable number of the same respondents simultaneously agreed that they needed further support to understand and use the system. Perceived ease of use differed significantly by region (F(3,149) = 5.67, p = 0.001), with the Western region highest (M = 4.2, SD = 0.6) and the Northern region lowest (M = 3.5, SD = 0.8); flexibility (p = 0.037) and acceptability (p = 0.018) followed the same gradient. Three qualitative themes explained the competency gap: insufficient practical focus in training, language and literacy barriers, and the absence of continuous post-rollout support. CONCLUSIONS Conclusions. eCBDS is broadly acceptable and easy to learn, but user competency is shallow and unevenly distributed, and perceived usability tracks geography rather than software design. A national usability claim for eCBDS is not currently defensible. Practical, locally-languaged and recurrent competency-based training is required to convert acceptability into proficiency. Keywords: eCBDS; DHIS2; usability; user competency; digital health; health workforce; public health surveillance; Sierra Leone; low- and middle-income countries CLINICALTRIAL NOT APPLICABLE