Assessing the potential utility of large language models for assisting community health workers: protocol for a prospective, observational study in Rwanda

Introduction Community health workers (CHWs) are
critical to healthcare delivery in low-resource settings but
often lack formal clinical training, limiting their decisionmaking. Large language models (LLMs) could provide
real-time, context-specific support to improve referrals
and management plans. This study aims to evaluate the
potential utility of LLMs in assisting CHW decision-making
in Rwanda.
Methods and analysis This is a prospective,
observational study conducted in Nyabihu and Musanze
districts, Rwanda. Audio recordings of CHW-patient
consultations will be transcribed and analysed by an
LLM to generate referral decisions, differential diagnoses
and management plans. These outputs, alongside
CHW decisions, will be evaluated against a clinical
expert panel’s consensus. The primary outcome is the
appropriateness of referral decisions. Secondary outcomes
include diagnostic accuracy, management plan quality,
and patient and user perceptions to ambient recording of
consultations. Sample size is set at 800 consultations (400
per district), powered to detect a 15–20 percentage point
improvement in referral appropriateness.
Ethics and dissemination Ethical approval has been
obtained from the Rwandan National Ethics Committee
(RNEC) (Ref number: RNEC 853/2025) in June 2025,
recruitment started in July 2025 and results are expected
in late 2025. Results will be disseminated via stakeholder
meetings, academic conferences and peer-reviewed
publication.