Adapting and Validating Tools to Assess the Usability and Acceptability of mHealth Tools Among Community Health Workers in Rural Settings: Development and Usability Study

Background Mobile health (mHealth) apps are increasingly leveraged to support community health workers (CHWs) in delivering high-quality care, particularly in low- and middle-income countries. However, despite the proliferation of mHealth tools, few have been implemented at scale, partly due to limited attention to usability and acceptability among end users. In sub-Saharan Africa, mHealth tools designed for CHWs often lack systematic evaluation using validated instruments tailored to local contexts. Without such assessments, it is difficult to ensure that these tools can be integrated effectively into CHW workflows and scaled sustainably. Objective This study aimed to adapt and validate existing mHealth usability and acceptability assessment tools to be contextually appropriate for CHWs in Rwanda. Specifically, we sought to ensure contextual appropriateness for CHWs supporting postoperative home follow-up for women after cesarean delivery. The resulting tool was designed for use in an implementation study of a novel CHW-led mHealth app. Methods This study was conducted in the Kirehe district, Rwanda, from October 2022 to March 2023. We adapted 2 established tools—the mHealth App Usability Questionnaire and selected items from the Practitioner Opinion (Acceptability) Scale—and added new items that reflect core functions of the CHW-focused mHealth app. All items were translated into Kinyarwanda and simplified to align with CHWs’ educational levels. We conducted a three-stage validation that consisted of (1) content validity testing with 8 local and international experts using a recommended content validity index threshold of >0.78; (2) face validity testing with 10 CHWs using a recommended face validity index threshold of ≥0.60; and (3) reliability testing using responses from 30 CHWs, with a Cronbach α coefficient of ≥0.70 indicating acceptable internal consistency. Results Of the 25 items assessed, 22 (88%) achieved a content validity index score of >0.78 for both clarity and relevance. The face validity index across all 22 items was 0.991, indicating strong comprehensibility and relevance to CHWs. Internal consistency was high: the Cronbach α was 0.86 for the mHealth App Usability Questionnaire items, 0.73 for the Practitioner Opinion (Acceptability) Scale items, and 0.87 for the newly developed questions. The final tool—named the Community Health Worker mHealth Usability and Acceptability Assessment Tool—included 22 items with strong content validity, face validity, and internal reliability. Conclusions This study presents a rigorously adapted and validated tool for assessing mHealth usability and acceptability among CHWs in Rwanda. The Community Health Worker mHealth Usability and Acceptability Assessment Tool can guide future evaluations of mHealth interventions in similar contexts and serve as a model for localizing mHealth assessment tools in low- and middle-income country settings to ensure fit-for-purpose implementation.