Dataset Information

Assessing the assessment in emergency care training.

ABSTRACT:

Objective

Each year over 1.5 million health care professionals attend emergency care courses. Despite high stakes for patients and extensive resources involved, little evidence exists on the quality of assessment. The aim of this study was to evaluate the validity and reliability of commonly used formats in assessing emergency care skills.

Methods

Residents were assessed at the end of a 2-week emergency course; a subgroup was videotaped. Psychometric analyses were conducted to assess the validity and inter-rater reliability of the assessment instrument, which included a checklist, a 9-item competency scale and a global performance scale.

Results

A group of 144 residents and 12 raters participated in the study; 22 residents were videotaped and re-assessed by 8 raters. The checklists showed limited validity and poor inter-rater reliability for the dimensions "correct" and "timely" (ICC?=?.30 and.39 resp.). The competency scale had good construct validity, consisting of a clinical and a communication subscale. The internal consistency of the (sub)scales was high (??=?.93/.91/.86). The inter-rater reliability was moderate for the clinical competency subscale (.49) and the global performance scale (.50), but poor for the communication subscale (.27). A generalizability study showed that for a reliable assessment 5-13 raters are needed when using checklists, and four when using the clinical competency scale or the global performance scale.

Conclusions

This study shows poor validity and reliability for assessing emergency skills with checklists but good validity and moderate reliability with clinical competency or global performance scales. Involving more raters can improve the reliability substantially. Recommendations are made to improve this high stakes skill assessment.

SUBMITTER: Dankbaar ME

PROVIDER: S-EPMC4270684 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing the assessment in emergency care training.

Dankbaar Mary E W ME Stegers-Jager Karen M KM Baarveld Frank F Merrienboer Jeroen J G van JJ Norman Geoff R GR Rutten Frans L FL van Saase Jan L C M JL Schuit Stephanie C E SC

PloS one 20141218 12

<h4>Objective</h4>Each year over 1.5 million health care professionals attend emergency care courses. Despite high stakes for patients and extensive resources involved, little evidence exists on the quality of assessment. The aim of this study was to evaluate the validity and reliability of commonly used formats in assessing emergency care skills.<h4>Methods</h4>Residents were assessed at the end of a 2-week emergency course; a subgroup was videotaped. Psychometric analyses were conducted to ass ...[more]

PMID: 25521702

Similar Datasets

Project description:Training healthcare providers in Emergency Obstetric Care (EmOC) has been shown to be effective in improving their capacity to provide this critical care package for mothers and babies. However, little is known about the costs and cost-effectiveness of such training. Understanding costs and cost-effectiveness is essential in guaranteeing value-for-money in healthcare spending. This study systematically reviewed the available literature on cost and cost-effectiveness of EmOC trainings.Peer-reviewed and grey literature was searched for relevant papers published after 1990. Studies were included if they described an economic evaluation of EmOC training and the training cost data were available. Two reviewers independently searched, screened, and selected studies that met the inclusion criteria, with disagreements resolved by a third reviewer. Quality of studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards statement. For comparability, all costs in local currency were converted to International dollar (I$) equivalents using purchasing power parity conversion factors. The cost per training per participant was calculated. Narrative synthesis was used to summarise the available evidence on cost effectiveness.Fourteen studies (five full and nine partial economic evaluations) met the inclusion criteria. All five and two of the nine partial economic evaluations were of high quality. The majority of studies (13/14) were from low- and middle-income countries. Training equipment, per diems and resource person allowance were the most expensive components. Cost of training per person per day ranged from I$33 to I$90 when accommodation was required and from I$5 to I$21 when training was facility-based. Cost-effectiveness of training was assessed in 5 studies with differing measures of effectiveness (knowledge, skills, procedure cost and lives saved) making comparison difficult.Economic evaluations of EmOC training are limited. There is a need to scale-up and standardise processes that capture both cost and effectiveness of training and to agree on suitable economic evaluation models that allow for comparability across settings.PROSPERO_CRD42016041911 .

Project description:ObjectivesThe primary objective was to survey pediatric emergency medicine (PEM) leaders and fellows regarding point-of-care ultrasound (POCUS) training in PEM fellowship programs, including teaching methods, training requirements, and applications taught. Secondary objectives were to compare fellows' and program leaders' perceptions of fellow POCUS competency and training barriers.MethodsThis was a cross-sectional survey of U.S. PEM fellows and fellowship program leaders of the 78 fellowship programs using two online group-specific surveys exploring five domains: program demographics; training strategies and requirements; perceived competency; barriers, strengths, and weaknesses of POCUS training; and POCUS satisfaction.ResultsEighty-three percent (65/78) of programs and 53% (298/558) of fellows responded. All participating PEM fellowship programs included POCUS training in their curriculum. Among the 65 programs, 97% of programs and 92% of programs utilized didactics and supervised scanning shifts as educational techniques, respectively. Sixty percent of programs integrated numerical benchmarks and 49% of programs incorporated real-time, hands-on demonstration as training requirements. Of the 19 POCUS applications deemed in the literature as core requirements for fellows, at least 75% of the 298 fellows reported training in 13 of those applications. Although less than half of fellows endorsed competency for identifying intussusception, ultrasound-guided pericardiocentesis, and transvaginal pregnancy evaluation, a higher proportion of leaders reported fellows as competent for these applications (40% vs. 68%, p ≤ 0.001; 21% vs. 39%, p = 0.003; and 21% vs. 43%, p ≤ 0.001). Forty-six percent of fellows endorsed a lack of PEM POCUS evidence as a training barrier compared to 31% of leaders (p = 0.02), and 39% of leaders endorsed a lack of local financial support as a training barrier compared to 23% of fellows (p = 0.01).ConclusionsAlthough most PEM fellowship programs provide POCUS training, there is variation in content and requirements. Training does conform to many of the expert recommended guidelines; however, there are some discrepancies and perceived barriers to POCUS training remain.

Dataset Information

Assessing the assessment in emergency care training.

Objective

Methods

Results

Conclusions

Publications

Assessing the assessment in emergency care training.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets