Trial Design

Eight U.S. medical schools were pair-matched and randomized to an MME or TE curriculum starting in the first year (MS1). The outcome was demonstration of WMC skills in the third year (MS3) assessed through a weight management-specific Objective Structured Clinical Examination (OSCE), and self-reported skills in performing the 5As. We compared scores for MS3 students in MME and TE schools.

Participating Schools

Schools met the following criteria: 1) at least 90 MS1s; 2) ≤4 hours of WMC education during the first 3 years of medical school curriculum; 3) offer a family medicine or internal medicine clerkship for MS3s; and 4) include an evaluation of MS3s using a standard OSCE and willing to include the study WMC case. Institutional review boards at each participating medical school approved the study. The U.S. schools were private (N=4) or public (N=4) in geographically diverse regions.

Pair-Matching and Randomization

Before randomization, principal investigators at each of the eight schools were surveyed about the number of hours of curricula addressing behavior change and WMC during MS1–MS3 in their medical schools ²² . Questions on this survey were based on content and format from previous research. In addition, a Matching and Randomization survey was administered to the Class of 2017 in both TE and MME schools which assessed various variables, including student self-perceived WMC skills (16 items; responses 1=not at all skilled, to 5=very skilled ) ²² . Schools were stratified into high (four schools) or low (four schools) and these pair-matched schools were randomly assigned to MME or TE.

Educational Interventions

Outcome Measures

We assessed WMC skills using an OSCE, a standard objective method to evaluate medical students in the U.S. The WMC OSCE case and associated checklist were developed by the investigators specifically for this study based on the 5As counseling approach recommended by the USPSTF. ^{22, 25} OSCEs were implemented within the context of existing clinical skills assessments, videotaped, then scored by a trained rater blinded to the student’s school. The 15-item behavior checklist assessed multiple aspects of each 5A behavior, scored Yes/No (range of 0-15) (Table 1). A ‘gold-standard’ rater who was extensively trained by the checklist developers reviewed and coded a random sample of 10% of all completed checklists. The discrepancy rate between coders and the gold-standard rater on this 10% random sample was 3.4% which was below our pre-determined 5% threshold.

We also assessed WMC skills using a subjective measure of student self-reported perceived skill level for each of the 16 5A items assessed in the OSCE, using a 4-category Likert scale (Table 2). Students completed a baseline survey as MS1s and a follow-up survey as MS3s. Total score was calculated as the average of the16 items.

Intervention Tracking and Exposure Measures

Study coordinators tracked each student’s participation in the web course, role-play, and web-patient encounter. The total MME intervention exposure received per student was computed as the total number of components completed (0–3). MS3s at MME schools also completed a survey on participation in the Web course and role-play regarding WMC, The self-reported total intervention exposure was computed as the sum of students receiving each of the MME components (range 0–3).

Statistical Analysis

Total OSCE case scores were analyzed using a two-stage mixed model analysis of variance (ANCOVA) ²⁶ . In stage 1, 16 (8 schools × 2 timepoints) means were estimated for student-level OSCEs. In stage 2, the eight post-intervention school means were regressed on randomization assignment and the corresponding pre-intervention school mean, adjusting the MME versus TE comparison for baseline school means and accounting for within-school correlation. In parallel, we estimated binomial logistic regression models for each of the OSCE items (completed versus not completed) as a function of randomization assignment with school as a random effect. ²⁷ Students in the MME intervention and students in the TE intervention (referred to below as MME students and TE students, respectively) were compared regarding perceived WMC skills using linear mixed modeling ^{28, 29} for total score and binomial logistic regression for each of the 16 individual skills, with randomization assignment as a predictor and a random effect for school, adjusting for student’s MS1 score. In addition to our primary intention to treat analyses, we estimated associations of WMC training (“exposure”) with outcomes. For MME students, separate models were estimated with participation in each of the intervention components and number of intervention components as predictors. Combining MME and TE students, we estimated associations of outcomes with self-reported hours of WMC learning in MS1, MS2, and MS3. MME and TE students were also compared regarding self-reported hours using ordinal logistic regression with school as a random effect. To assess any impact of missing OSCE scores (29.9%, 390/1305), we multiply imputed missing OSCEs ^{30, 31} using IVEware as a function of: randomization assignment, school, baseline school mean OSCE, gender, race, and MS1 and MS3 measures of attitudes towards WMC, obesity bias ³² , confidence in providing WMC, and perceived WMC skills. Results regarding MME-TE differences were consistent with those presented here (data not shown). Analyses used SAS 9.4 ³³ and Stata 14.2 ³⁴

Participating Students

At MME schools, 629 MS3s were eligible; of these, 479 (76.2%) completed a third-year post-intervention OSCE, and 474 (76.3%) completed the follow-up survey. At TE schools, 687 MS3s were eligible; of these, 436 (63.5%) completed a third-year OSCE, and 432 (62.9%) completed the follow-up survey. Figure 1 shows reasons for missing outcomes. Among students providing OSCE data or self-reported WMC skills, MME and TE students were similar in age, race, Hispanic ethnicity, and gender. Students were ~24 years old, 51% were female, 74% identified as White, 6% as African American, 15% as Asian, 5% as Multi-racial, 1% as ‘other’ race, and 5% as Hispanic. Students reported their intended career as follows: primary care (24.1%), medical specialty (53.7%), other (11.4%), and undecided (10.8%)

OSCE Outcomes

On average, MS3 MME students completed 8.7 of the 15 5A items on the case checklist (SE 0.11) versus 8.2 (SE 0.11) for TE students (p=0.16) (Table 1). MME students outperformed TE students on 12 of 15 items (p-values ranging from .09 to .91) . In both MME and TE schools, >85% of students reviewed medical risk factors and co-existing conditions of obesity, asked the patient standard questions about diet and physical activity, discussed the patient’s perceived barriers and concerns with losing weight/maintaining weight loss, and discussed behavior modifications to achieve their goals.

Self-Reported WMC Skills

At post-intervention, mean total score (SD) was slightly higher for MME students (3.18) than TE students (3.09) (p=0.25, scale of 1 to 4). Although only one item-specific difference was statistically significant, a higher percentage of MME versus TE students reported feeling moderately/very skilled in 14 of the 16 WMC skills assessed (Table 2), with largest absolute differences for discussing the health benefits of losing weight, recommending or referring the patient to weight management resources, and advising weight loss based on their personal health information.

WMC training in MME and TE students

At MME schools, more than 2/3 of students completed all three intervention-delivered activities (Table 3). MME students reported significantly more preclinical time in WMC during the MS1 and MS2 years than TE students, however both groups spent an equivalent amount of time during MS3.

Associations of WMC training with outcomes

At MME schools, student participation in the three intervention components was positively associated with total OSCE score, with the largest difference for the web course (0.43 items difference on average), although the associations were not statistically significant (Table 4). MME students who completed two of the three components had the highest mean OSCE score at 9.02 (standard error 0.39), and those completing no components had the lowest mean OSCE score at 8.22 (standard error 0.44), although differences were not statistically significant (p=0.28). MME student participation in the three intervention components also was positively associated with perceived WMC skills, with statistically significant associations for all but web course participation. Among students at both MME and TE schools, time spent in WMC learning activities in the first two years ranging from 1 (<30 minutes) to 8 (>5 hours) was significantly positively associated with total OSCE score and total perceived WMC skills; an increase of 1 category (e.g., <30 minutes to 30-59 minutes) was associated with an average increase of 0.08 in the OSCE score and of 0.04 in total perceived WMC skills. The corresponding associations with time in MS3 year also were positive but smaller and not statistically significant for the OSCE score.

Strengths and Limitations

The major strengths of this study were the randomized design, the participation and excellent response rates of eight schools nationwide, high-fidelity across multiple and diverse medical schools, relatively strong student participation in all MME components, and rigorous evaluation via two time-tested assessments (OSCE and self-report). We also chose schools that had ≤4 hours of WMC education to attract schools without a WMC-rich curriculum.

Because only eight U.S. medical schools were involved, the study may have limited generalizability. A second concern is that questions do not capture the full breadth of student performance,for example, students may conduct routine WMC but are unaware of doing this for patients with co-existing conditions like diabetes and heart disease. Third, while there is some concern that one-third of students did not complete follow-up assessments, most missing data in both arms was attributed to academic reasons (e.g., dismissed / left school, schedule shift, delayed by a semester,change of campus, part of MD/PhD program, leave of absence). Finally, while OSCE’s are widely used in medical education, performance on a single OSCE in this case as part of the existing curriculum provides only a snapshot of a student’s skill in WMC.

Conclusion

This study is consistent with our prior trial to improve medical students’ tobacco treatment counseling skills. ³⁵ In both cases, students in TE schools received enough training to possibly dilute an effect from the MME curriculum. Unlike our prior study, where students performed well at advising patients to stop smoking, this study showed that students struggle with advising patients about diet, exercise, and the benefits of weight loss and weight management.

A systematic review in 2020 of weight-related communication trainings for physicians concluded that physician trainings should be grounded in a theoretical framework and emphasize patient-centered communication—experiential learning and skill development should be central components since they appear to improve physician outcomes ³⁶ . Our results are in line with these recommendations. Future WMC curricula must improve instruction in nutrition and exercise, along with behavioral change counseling skills. Experiential education to provide knowledge and build confidence in delivering counseling should start early in medical school and be boosted throughout the four years of training.

We acknowledge all school site PIs, research coordinators, evaluators, directors of standardized patients, medical directors of clinical education and simulation center, research assistants, support staff, participating medical students, course directors, clerkship directors, preceptors and our weight bias consultant who have and will implement and participate in the study.