Dataset Information

DOPS (Direct Observation of Procedural Skills) in undergraduate skills-lab: Does it work? Analysis of skills-performance and curricular side effects.

ABSTRACT: Sufficient teaching and assessing clinical skills in the undergraduate setting becomes more and more important. In a surgical skills-lab course at the Medical University of Innsbruck fourth year students were teached with DOPS (direct observation of procedural skills). We analyzed whether DOPS worked or not in this setting, which performance levels could be reached compared to tutor teaching (one tutor, 5 students) and which curricular side effects could be observed.In a prospective randomized trial in summer 2013 (April - June) four competence-level-based skills were teached in small groups during one week: surgical abdominal examination, urethral catheterization (phantom), rectal-digital examination (phantom), handling of central venous catheters. Group A was teached with DOPS, group B with a classical tutor system. Both groups underwent an OSCE (objective structured clinical examination) for assessment. 193 students were included in the study. Altogether 756 OSCE´s were carried out, 209 (27,6%) in the DOPS- and 547 (72,3%) in the tutor-group.Both groups reached high performance levels. In the first month there was a statistically significant difference (p<0,05) in performance of 95% positive OSCE items in the DOPS-group versus 88% in the tutor group. In the following months the performance rates showed no difference anymore and came to 90% in both groups. In practical skills the analysis revealed a high correspondence between positive DOPS (92,4%) and OSCE (90,8%) results.As shown by our data DOPS furnish high performance of clinical skills and work well in the undergraduate setting. Due to the high correspondence of DOPS and OSCE results DOPS should be considered as preferred assessment tool in a students skills-lab. The approximation of performance-rates within the months after initial superiority of DOPS could be explained by an interaction between DOPS and tutor system: DOPS elements seem to have improved tutoring and performance rates as well. DOPS in students 'skills-lab afford structured feedback and assessment without increased personnel and financial resources compared to classic small group training.In summary, this study shows that DOPS represent an efficient method in teaching clinical skills. Their effects on didactic culture reach beyond the positive influence of performance rates.

SUBMITTER: Profanter C

PROVIDER: S-EPMC4606486 | biostudies-other | 2015

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

DOPS (Direct Observation of Procedural Skills) in undergraduate skills-lab: Does it work? Analysis of skills-performance and curricular side effects.

Profanter Christoph C Perathoner Alexander A

GMS Zeitschrift fur medizinische Ausbildung 20151015 4

<h4>Objective</h4>Sufficient teaching and assessing clinical skills in the undergraduate setting becomes more and more important. In a surgical skills-lab course at the Medical University of Innsbruck fourth year students were teached with DOPS (direct observation of procedural skills). We analyzed whether DOPS worked or not in this setting, which performance levels could be reached compared to tutor teaching (one tutor, 5 students) and which curricular side effects could be observed.<h4>Methods ...[more]

PMID: 26483858

Similar Datasets

Project description:BACKGROUND:Validated competency assessment tools and the data supporting milestone development during gastroscopy training are lacking. We aimed to assess the validity of the formative direct observation of procedural skills (DOPS) assessment tool in diagnostic gastroscopy and study competency development using DOPS. METHODS:This was a prospective multicentre (N = 275) analysis of formative gastroscopy DOPS assessments. Internal structure validity was tested using exploratory factor analysis and reliability estimated using generalisability theory. Item and global DOPS scores were stratified by lifetime procedure count to define learning curves, using a threshold determined from receiver operator characteristics (ROC) analysis. Multivariable binary logistic regression analysis was performed to identify independent predictors of DOPS competence. RESULTS:In total, 10086 DOPS were submitted for 987 trainees. Exploratory factor analysis identified three distinct item groupings, representing 'pre-procedure', 'technical', and 'post-procedure non-technical' skills. From generalisability analyses, sources of variance in overall DOPS scores included trainee ability (31%), assessor stringency (8%), assessor subjectivity (18%), and trainee case-to-case variation (43%). The combination of three assessments from three assessors was sufficient to achieve the reliability threshold of 0.70. On ROC analysis, a mean score of 3.9 provided optimal sensitivity and specificity for determining competency. This threshold was attained in the order of 'pre-procedure' (100-124 procedures), 'technical' (150-174 procedures), 'post-procedure non-technical' skills (200-224 procedures), and global competency (225-249 procedures). Higher lifetime procedure count, DOPS count, surgical trainees and assessors, higher trainee seniority, and lower case difficulty were significant multivariable predictors of DOPS competence. CONCLUSION:This study establishes milestones for competency acquisition during gastroscopy training and provides validity and reliability evidence to support gastroscopy DOPS as a competency assessment tool.

Project description:INTRODUCTION:Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences. METHODS:We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr's adaptation of Kirkpatrick's four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher's exact test for count data. RESULTS:We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners' reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees' performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, "quality" and "participant responsiveness", to be associated with the educational impact. CONCLUSIONS:Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.

Project description:BackgroundMedical educators struggle to incorporate socio-cultural topics into crowded curricula. The "continuum of learning" includes undergraduate and graduate medical education. Utilizing an exemplar socio-cultural topic, we studied the feasibility of achieving expert consensus among two groups of faculty (experts in medical education and experts in social determinants of health) on which aspects of the topic could be taught during undergraduate versus graduate medical education.MethodsA modified Delphi method was used to generate expert consensus on which learning objectives of social determinants of health are best taught at each stage of medical education. Delphi respondents included experts in medical education or social determinants of health. A survey was created using nationally published criteria for social determinants of health learning objectives. Respondents were asked 1) which learning objectives were necessary for every physician (irrespective of specialty) to develop competence upon completion of medical training and 2) when the learning objective should be taught. Respondents were also asked an open-ended question on how they made the determination of when in the medical education continuum the learning objective should be taught.Results26 out of 55 experts (13 social determinants of health and 13 education experts) responded to all 3 Delphi rounds. Experts evaluated a total of 49 learning objectives and were able to achieve consensus for at least one of the two research questions for 45 of 49 (92%) learning objectives. 50% more learning objectives reached consensus for inclusion in undergraduate (n = 21) versus graduate medical education (n = 14).ConclusionsA modified Delphi technique demonstrated that experts could identify key learning objectives of social determinants of health needed by all physicians and allocate content along the undergraduate and graduate medical education continuum. This approach could serve as a model for similar socio-cultural content. Future work should employ a qualitative approach to capture principles utilized by experts when making these decisions.

Dataset Information

DOPS (Direct Observation of Procedural Skills) in undergraduate skills-lab: Does it work? Analysis of skills-performance and curricular side effects.

Publications

DOPS (Direct Observation of Procedural Skills) in undergraduate skills-lab: Does it work? Analysis of skills-performance and curricular side effects.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets