Dataset Information

Using prescribing very short answer questions to identify sources of medication errors: a prospective study in two UK medical schools.

ABSTRACT:

Objective

To assess the utility and ability of the novel prescribing very short answer (VSA) question format to identify the sources of undergraduate prescribing errors when compared with the conventional single best answer (SBA) question format and assess the acceptability of machine marking prescribing VSAs.

Design

A prospective study involving analysis of data generated from a pilot two-part prescribing assessment.

Setting

Two UK medical schools.

Participants

364 final year medical students took part. Participation was voluntary. There were no other inclusion or exclusion criteria.

Outcomes

(1) Time taken to mark and verify VSA questions (acceptability), (2) differences between VSA and SBA scores, (3) performance in VSA and (4) SBA format across different subject areas and types of prescribing error made in the VSA format.

Results

18?200 prescribing VSA questions were marked and verified in 91?min. The median percentage score for the VSA test was significantly lower than the SBA test (28% vs 64%, p<0.0001). Significantly more prescribing errors were detected in the VSA format than the SBA format across all domains, notably in prescribing insulin (96.4% vs 50.3%, p<0.0001), fluids (95.6% vs 55%, p<0.0001) and analgesia (85.7% vs 51%, p<0.0001). Of the incorrect VSA responses, 33.1% were due to the medication prescribed, 6.0% due to the dose, 1.4% due to the route and 4.8% due to the frequency.

Conclusions

Prescribing VSA questions represent an efficient tool for providing detailed insight into the sources of significant prescribing errors, which are not identified by SBA questions. This makes the prescribing VSA a valuable formative assessment tool to enhance students' skills in safe prescribing and to potentially reduce prescribing errors.

SUBMITTER: Sam AH

PROVIDER: S-EPMC6629393 | biostudies-literature | 2019 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using prescribing very short answer questions to identify sources of medication errors: a prospective study in two UK medical schools.

Sam Amir H AH Fung Chee Yeen CY Wilson Rebecca K RK Peleva Emilia E Kluth David C DC Lupton Martin M Owen David R DR Melville Colin R CR Meeran Karim K

BMJ open 20190709 7

<h4>Objective</h4>To assess the utility and ability of the novel prescribing very short answer (VSA) question format to identify the sources of undergraduate prescribing errors when compared with the conventional single best answer (SBA) question format and assess the acceptability of machine marking prescribing VSAs.<h4>Design</h4>A prospective study involving analysis of data generated from a pilot two-part prescribing assessment.<h4>Setting</h4>Two UK medical schools.<h4>Participants</h4>364 ...[more]

PMID: 31289084

Similar Datasets

Project description:OBJECTIVES:The study aimed to compare candidate performance between traditional best-of-five single-best-answer (SBA) questions and very-short-answer (VSA) questions, in which candidates must generate their own answers of between one and five words. The primary objective was to determine if the mean positive cue rate for SBAs exceeded the null hypothesis guessing rate of 20%. DESIGN:This was a cross-sectional study undertaken in 2018. SETTING:20 medical schools in the UK. PARTICIPANTS:1417 volunteer medical students preparing for their final undergraduate medicine examinations (total eligible population across all UK medical schools approximately 7500). INTERVENTIONS:Students completed a 50-question VSA test, followed immediately by the same test in SBA format, using a novel digital exam delivery platform which also facilitated rapid marking of VSAs. MAIN OUTCOME MEASURES:The main outcome measure was the mean positive cue rate across SBAs: the percentage of students getting the SBA format of the question correct after getting the VSA format incorrect. Internal consistency, item discrimination and the pass rate using Cohen standard setting for VSAs and SBAs were also evaluated, and a cost analysis in terms of marking the VSA was performed. RESULTS:The study was completed by 1417 students. Mean student scores were 21 percentage points higher for SBAs. The mean positive cue rate was 42.7% (95% CI 36.8% to 48.6%), one-sample t-test against ≤20%: t=7.53, p<0.001. Internal consistency was higher for VSAs than SBAs and the median item discrimination equivalent. The estimated marking cost was £2655 ($3500), with 24.5 hours of clinician time required (1.25 s per student per question). CONCLUSIONS:SBA questions can give a false impression of students' competence. VSAs appear to have greater authenticity and can provide useful information regarding students' cognitive errors, helping to improve learning as well as assessment. Electronic delivery and marking of VSAs is feasible and cost-effective.

Project description:OBJECTIVE:Clinicians using clinical decision support (CDS) to prescribe medications have an obligation to ensure that prescriptions are safe. One option is to verify the safety of prescriptions if there is uncertainty, for example, by using drug references. Supervisory control experiments in aviation and process control have associated errors, with reduced verification arising from overreliance on decision support. However, it is unknown whether this relationship extends to clinical decision-making. Therefore, we examine whether there is a relationship between verification behaviors and prescribing errors, with and without CDS medication alerts, and whether task complexity mediates this. METHODS:A total of 120 students in the final 2 years of a medical degree prescribed medicines for patient scenarios using a simulated electronic prescribing system. CDS (correct, incorrect, and no CDS) and task complexity (low and high) were varied. Outcomes were omission (missed prescribing errors) and commission errors (accepted false-positive alerts). Verification measures were access of drug references and view time percentage of task time. RESULTS:Failure to access references for medicines with prescribing errors increased omission errors with no CDS (high-complexity: χ 2(1) = 12.716; p < 0.001) and incorrect CDS (Fisher's exact; low-complexity: p = 0.002; high-complexity: p = 0.001). Failure to access references for false-positive alerts increased commission errors (low-complexity: χ 2(1) = 16.673, p < 0.001; high-complexity: χ 2(1) = 18.690, p < 0.001). Fewer participants accessed relevant references with incorrect CDS compared with no CDS (McNemar; low-complexity: p < 0.001; high-complexity: p < 0.001). Lower view time percentages increased omission (F(3, 361.914) = 4.498; p = 0.035) and commission errors (F(1, 346.223) = 2.712; p = 0.045). View time percentages were lower in CDS-assisted conditions compared with unassisted conditions (F(2, 335.743) = 10.443; p < 0.001). DISCUSSION:The presence of CDS reduced verification of prescription safety. When CDS was incorrect, reduced verification was associated with increased prescribing errors. CONCLUSION:CDS can be incorrect, and verification provides one mechanism to detect errors. System designers need to facilitate verification without increasing workload or eliminating the benefits of correct CDS.

Project description:Multiple choice questions (MCQs) offer high reliability and easy machine-marking, but allow for cueing and stimulate recognition-based learning. Very short answer questions (VSAQs), which are open-ended questions requiring a very short answer, may circumvent these limitations. Although VSAQ use in medical assessment increases, almost all research on reliability and validity of VSAQs in medical education has been performed by a single research group with extensive experience in the development of VSAQs. Therefore, we aimed to validate previous findings about VSAQ reliability, discrimination, and acceptability in undergraduate medical students and teachers with limited experience in VSAQs development. To validate the results presented in previous studies, we partially replicated a previous study and extended results on student experiences. Dutch undergraduate medical students (n = 375) were randomized to VSAQs first and MCQs second or vice versa in a formative exam in two courses, to determine reliability, discrimination, and cueing. Acceptability for teachers (i.e., VSAQ review time) was determined in the summative exam. Reliability (Cronbach's α) was 0.74 for VSAQs and 0.57 for MCQs in one course. In the other course, Cronbach's α was 0.87 for VSAQs and 0.83 for MCQs. Discrimination (average Rir) was 0.27 vs. 0.17 and 0.43 vs. 0.39 for VSAQs vs. MCQs, respectively. Reviewing time of one VSAQ for the entire student cohort was ±2 minutes on average. Positive cueing occurred more in MCQs than in VSAQs (20% vs. 4% and 20.8% vs. 8.3% of questions per person in both courses). This study validates the positive results regarding VSAQs reliability, discrimination, and acceptability in undergraduate medical students. Furthermore, we demonstrate that VSAQ use is reliable among teachers with limited experience in writing and marking VSAQs. The short learning curve for teachers, favourable marking time and applicability regardless of the topic suggest that VSAQs might also be valuable beyond medical assessment.

Project description:ImportanceCritically ill children presenting to emergency departments (EDs) in non-children's hospitals are at high risk for experiencing medical errors, including medication errors. Video telemedicine consultations with pediatric specialists have the potential to reduce the risk of medication errors beyond the current standard of care, telephone consultations.ObjectiveTo compare the rates of ED physician-related medication errors among critically ill children randomized to receive either video telemedicine or telephone consultations.Design, setting, and participantsThis cluster randomized, unbalanced crossover trial was conducted at 15 community EDs in northern California between September 2014 and March 2018. Analyses were conducted from May 2022 to January 2023. Participants included acutely ill children younger than 15 years presenting to a participating ED.InterventionsParticipating EDs were randomized to use video telemedicine or telephone for consultations with pediatric critical care physicians according to 1 of 4 unbalanced (3 telemedicine to 1 telephone) crossover treatment assignment sequences.Main outcomes and measuresPharmacists reviewed medical records to document physician-related medication errors using a previously validated instrument. Multilevel logistic regression analyses were performed to create models with the medication order as the unit of analysis and adjusting for age, the log-transformed Revised Pediatric Emergency Assessment Tool score, and hospital study period.ResultsA total of 696 patient encounters were included in the trial (mean [SD] age, 4.2 [4.6] years; median [IQR] age, 2.1 [0.5-2.1] years; 304 female [43.7%]), with 537 patient encounters (77.2%) assigned to video telemedicine and 159 patient encounters (22.8%) assigned to telephone. At least 1 physician-related medication error occurred for 87 patients (12.5%), including 20 of 159 patients (12.6%) in the telephone cohort and 67 of 537 patients (12.5%) in the telemedicine cohort. Of the 2414 medication orders, errors occurred in 124 cases (5.1%), including 26 of 513 orders (5.1%) in the telephone cohort and 98 of 1901 orders (5.2%) in the telemedicine cohort. In the multivariable analysis, the adjusted odds ratio of experiencing a medication error among those assigned to telemedicine was 0.86 (95% CI, 0.49-1.52; P = .61).Conclusions and relevanceThis cluster randomized crossover trial found no statistically significant differences in physician-related medication errors between critically ill children assigned to receive telephone consultations vs video telemedicine consultations.Trial registrationClinicalTrials.gov Identifier: NCT02877810.

Project description:BACKGROUND: The huge amount of biomedical-molecular data increasingly produced is providing scientists with potentially valuable information. Yet, such data quantity makes difficult to find and extract those data that are most reliable and most related to the biomedical questions to be answered, which are increasingly complex and often involve many different biomedical-molecular aspects. Such questions can be addressed only by comprehensively searching and exploring different types of data, which frequently are ordered and provided by different data sources. Search Computing has been proposed for the management and integration of ranked results from heterogeneous search services. Here, we present its novel application to the explorative search of distributed biomedical-molecular data and the integration of the search results to answer complex biomedical questions. RESULTS: A set of available bioinformatics search services has been modelled and registered in the Search Computing framework, and a Bioinformatics Search Computing application (Bio-SeCo) using such services has been created and made publicly available at http://www.bioinformatics.deib.polimi.it/bio-seco/seco/. It offers an integrated environment which eases search, exploration and ranking-aware combination of heterogeneous data provided by the available registered services, and supplies global results that can support answering complex multi-topic biomedical questions. CONCLUSIONS: By using Bio-SeCo, scientists can explore the very large and very heterogeneous biomedical-molecular data available. They can easily make different explorative search attempts, inspect obtained results, select the most appropriate, expand or refine them and move forward and backward in the construction of a global complex biomedical query on multiple distributed sources that could eventually find the most relevant results. Thus, it provides an extremely useful automated support for exploratory integrated bio search, which is fundamental for Life Science data driven knowledge discovery.

Project description:It has been suggested that doctors in their first year of post-graduate training make a disproportionate number of prescribing errors.This study aimed to compare the prevalence of prescribing errors made by first-year post-graduate doctors with that of errors by senior doctors and non-medical prescribers and to investigate the predictors of potentially serious prescribing errors.Pharmacists in 20 hospitals over 7 prospectively selected days collected data on the number of medication orders checked, the grade of prescriber and details of any prescribing errors. Logistic regression models (adjusted for clustering by hospital) identified factors predicting the likelihood of prescribing erroneously and the severity of prescribing errors.Pharmacists reviewed 26,019 patients and 124,260 medication orders; 11,235 prescribing errors were detected in 10,986 orders. The mean error rate was 8.8 % (95 % confidence interval [CI] 8.6-9.1) errors per 100 medication orders. Rates of errors for all doctors in training were significantly higher than rates for medical consultants. Doctors who were 1 year (odds ratio [OR] 2.13; 95 % CI 1.80-2.52) or 2 years in training (OR 2.23; 95 % CI 1.89-2.65) were more than twice as likely to prescribe erroneously. Prescribing errors were 70 % (OR 1.70; 95 % CI 1.61-1.80) more likely to occur at the time of hospital admission than when medication orders were issued during the hospital stay. No significant differences in severity of error were observed between grades of prescriber. Potentially serious errors were more likely to be associated with prescriptions for parenteral administration, especially for cardiovascular or endocrine disorders.The problem of prescribing errors in hospitals is substantial and not solely a problem of the most junior medical prescribers, particularly for those errors most likely to cause significant patient harm. Interventions are needed to target these high-risk errors by all grades of staff and hence improve patient safety.

Dataset Information

Using prescribing very short answer questions to identify sources of medication errors: a prospective study in two UK medical schools.

Objective

Design

Setting

Participants

Outcomes

Results

Conclusions

Publications

Using prescribing very short answer questions to identify sources of medication errors: a prospective study in two UK medical schools.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets