Events
Public Oral Defense: Jacqueline L. Gauer
-
Virtual
A Mixed Methods Evaluation of the Reliability and Validity of an Entrustable Professional Activities-Based System for Assessing the Clinical Competence of Medical Students
Advisor: Stuart Yeh
Introduction: The question of how to assess the clinical competence of medical students has posed a challenge throughout the history of the field. Entrustable Professional Activities (EPAs) describe discrete activities that can be assessed in the workplace by a preceptor, who indicates the degree to which they “entrust” the student to perform the activity. Recently, the University of Minnesota Medical School (UMMS) implemented a system based on the Core Entrustable Professional Activities for Entering Residency (Core-EPAs) to assess the clinical skills of third-year medical students during their clinical clerkships. The purpose of this study was to evaluate the validity and reliability of this system, using quantitative data, qualitative data, and the integration of the two.
Method: This study employed a two-phase, sequential explanatory mixed methods design to obtain evidence regarding the reliability, predictive validity, construct validity, and face validity of the Core-EPA system. Quantitative EPA assessment data from AY 22-23 were analyzed via interrater agreement analysis, linear regression modeling to predict scores on an Objective Structured Clinical Examination (OSCE), and growth curve modeling. A purposive sample of eight students was selected from the quantitative data to describe their experiences via semi-structured interviews. The qualitative data were analyzed using a Reflexive Thematic Analysis framework.
Results: The interrater reliability analysis found that levels of interrater agreement were acceptable given the complexity of the clinical context (47.35% or 69.40% depending on the definition of interrater agreement), but with room for improvement. The linear regression analysis did not find convincing evidence that EPA ratings are predictive of OSCE scores. The growth curve analysis found that growth curves aligned with those expected by learning curve theory. The qualitative analysis generated five themes and two subthemes describing students’ experiences with the Core-EPA system, their perspectives on its validity and reliability including factors that contribute to those dimensions, and comparisons between the Core-EPA system and OSCEs.
Discussion: The findings of this study indicate that the Core-EPA system holds promise as a tool for assessing the clinical competence of medical students. Recommendations, limitations of the study, and ideas for next steps are described.
Photographs taken at the event may be used in University of Minnesota print and online publications, promotions, or shared with the CEHD community.
The University of Minnesota shall provide equal access to and opportunity in its programs, facilities, and employment without regard to race, color, creed, religion, national origin, gender, age, marital status, familial status, disability, public assistance status, membership or activity in a local commission created for the purpose of dealing with discrimination, veteran status, sexual orientation, gender identity, or gender expression. All are welcome.
The University of Minnesota campuses were built on the traditional homelands of the Dakota and Ojibwe, and scores of other Indigenous peoples who have walked on these lands from time immemorial. It is important to acknowledge the peoples on whose land we live, learn, and work as we seek to improve and strengthen our relations with our tribal nations. We also acknowledge that words are not enough. We must ensure that our institution provides support, resources, and programs that increase access to all aspects of higher education for our American Indian students, staff, faculty, and community members.