Correlation of Performance on ENTRUST and Traditional Oral Objective Structured Clinical Examination for High-Stakes Assessment in the College of Surgeons of East, Central, and Southern Africa

Background:
To address the global need for accessible evidence-based tools for competency-based education, we developed ENTRUST, an innovative online virtual patient simulation platform to author and securely deploy case scenarios to assess surgical decision-making competence.

Study Design:
In partnership with COSECSA, ENTRUST was piloted during the Membership of the College of Surgeons (MCS) 2021 examination. Examinees (n=110) completed the traditional 11-station oral OSCE, followed by three ENTRUST cases, authored to query similar clinical content of three corresponding OSCE cases. ENTRUST scores were analyzed for associations with MCS Exam outcome using independent sample t-tests. Correlation of ENTRUST scores to MCS Exam Percentage and OSCE Station Scores were calculated with Pearson correlations. Bivariate and multivariate analyses were performed to evaluate predictors of performance.

Results:
ENTRUST performance was significantly higher in examinees who passed the MCS Exam compared to those who failed (p<0.001). ENTRUST score was positively correlated with MCS Exam Percentage (p<0.001) and combined OSCE Station Scores (p<0.001). On multivariate analysis, there was a strong association between MCS Exam Percentage and ENTRUST Grand Total Score (p<0.001), Simulation Total Score (p=0.018), and Question Total Score (p<0.001). Age was a negative predictor for ENTRUST Grand Total and Simulation Total Score, but not for Question Total Score. Sex, native language status, and intended specialty were not associated with performance on ENTRUST.

Conclusion:
This study demonstrates feasibility and initial validity evidence for the use of ENTRUST in a high-stakes examination context for assessment of surgical decision-making. ENTRUST holds potential as an accessible learning and assessment platform for surgical trainees worldwide.