Dataset Information

Successful Implementation of a Direct Observation Program in an Ambulatory Block Rotation.

ABSTRACT:

Background?

Direct observation of clinical skills is a cornerstone of competency-based education and training. Ensuring direct observation in a consistent fashion has been a significant challenge for residency programs.

Objective?

The purpose of this study was to evaluate the effects of a novel evaluation system, designed to achieve ongoing direct observation of residents, examine changes in resident observation practices, and understand faculty attitudes toward direct observation and the evaluation system.

Methods?

Internal medicine residents on an ambulatory block rotation participated in a new evaluation system, which replaced a single end-of-rotation summative evaluation with 9 formative evaluations based on direct observation. Faculty received training in direct observation and use of the forms, and residents were given responsibility to collect 9 observations per rotation. Faculty members contacted residents at the beginning and middle of the rotation to ensure completion of the observations. Residents and faculty also completed postrotation surveys to gauge the impact of the new system.

Results?

A total of 507 patient encounters were directly observed, and 52 of 57 (91%) residents completed all 9 observations. Residents reported considerably more direct observation than prior to the intervention, and most reported changes to their clinical skills based on faculty feedback. Faculty reported improvements in their attitudes, increased their use of direct observation, and preferred the new system to the old one.

Conclusions?

A novel evaluation system replacing summative evaluations with multiple formative evaluations based on direct observation was successful in achieving high rates of observations, and improving faculty attitudes toward direct observation.

SUBMITTER: Smith J

PROVIDER: S-EPMC5319609 | biostudies-literature | 2017 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Successful Implementation of a Direct Observation Program in an Ambulatory Block Rotation.

Smith Jeremy J Jacobs Elizabeth E Li Zhanhai Z Vogelman Bennett B Zhao Yingqi Y Feldstein David D

Journal of graduate medical education 20170201 1

<h4>Background</h4>Direct observation of clinical skills is a cornerstone of competency-based education and training. Ensuring direct observation in a consistent fashion has been a significant challenge for residency programs.<h4>Objective</h4>The purpose of this study was to evaluate the effects of a novel evaluation system, designed to achieve ongoing direct observation of residents, examine changes in resident observation practices, and understand faculty attitudes toward direct observation a ...[more]

PMID: 28261405

Similar Datasets

Project description:BackgroundA key component of competency-based medical education is workplace-based assessment, which includes observation (direct or indirect) of residents. Direct observation has been emphasized as an ideal form of assessment yet challenges have been identified that may limit its adoption. At present, it remains unclear how often direct and indirect observation are being used within the clinical setting. The objective of this study was to describe patterns of observation in an emergency medicine competency-based program 2 years postimplementation.MethodsEmergency medicine residents (n = 19) recorded the type of observation they received (direct or indirect) following workplace-based entrustable professional activity (EPA) assessments from December 15, 2019, to April 30, 2020. Assessment forms were reviewed and analyzed to describe patters of observation.ResultsAssessments were collected on all 19 eligible residents (100% participation). A total of 1,070 EPA assessments were completed during the study period, of which 798 (74.6%) had the type of observation recorded. Of these recorded observations, 546 (68.4%) were directly observed and 252 (31.6%) were indirectly observed. The length of written comments contained within assessments following direct and indirect observation did not differ significantly. There was no significant association between resident gender and observation type or resident stage of training and observation type. Certain EPA assessments showed a clear preference toward either direct or indirect observation.ConclusionsTo the best of our knowledge, this study is the first to report patterns of observation in a competency-based residency program. The results suggest that direct observation can be quickly adopted as the primary means of workplace-based assessment. Indirect observation comprised a sizeable minority of observations and may be an underrecognized contributor to workplace-based assessment. The preference toward either direct or indirect observation for certain EPA assessments suggests that the entrustable professional activity itself may influence the type of observation.

Project description:Aim: As part of the MaReCuM model curriculum at Medical Faculty Mannheim Heidelberg University, a final year rotation in ambulatory care was implemented and augmented to include ambulatory care simulation. In this paper we describe this ambulatory care simulation, the designated competency-based learning objectives, and evaluate the educational effect of the ambulatory care simulation training. Method: Seventy-five final year medical students participated in the survey (response rate: 83%). The control group completed the ambulatory rotation prior to the implementation of the ambulatory care simulation. The experimental group was required to participate in the simulation at the beginning of the final year rotation in ambulatory care. A survey of both groups was conducted at the beginning and at the end of the rotation. The learning objectives were taken from the National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education (NKLM). Results: The ambulatory care simulation had no measurable influence on students' subjectively perceived learning progress, the evaluation of the ambulatory care rotation, or working in an ambulatory care setting. At the end of the rotation participants in both groups reported having gained better insight into treating outpatients. At the beginning of the rotation members of both groups assessed their competencies to be at the same level. The simulated ambulatory scenarios were evaluated by the participating students as being well structured and easy to understand. The scenarios successfully created a sense of time pressure for those confronted with them. The ability to correctly fill out a narcotic prescription form as required was rated significantly higher by those who participated in the simulation. Participation in the ambulatory care simulation had no effect on the other competencies covered by the survey. Discussion: The effect of the four instructional units comprising the ambulatory care simulation was not measurable due to the current form or the measurement point at the end of the 12-week rotation. The reasons for this could be the many and statistically elusive factors relevant to the individual and the wide variety among final year rotation placements, the late point in time of the final survey, and the selection of simulated scenarios. The course is slated to undergo specific further development and should be supplemented with additional learning opportunities to ensure that the main learning objectives are covered. The description of the teaching format is meant to contribute to the ongoing development of medical education with an emphasis on competency in the areas of ambulatory care, communication, prevention and health promotion.

Dataset Information

Successful Implementation of a Direct Observation Program in an Ambulatory Block Rotation.

Background?

Objective?

Methods?

Results?

Conclusions?

Publications

Successful Implementation of a Direct Observation Program in an Ambulatory Block Rotation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets