Dataset Information

Further Tests of the Utility of Integrated Speed-Accuracy Measures in Task Switching.

ABSTRACT: Speed and accuracy of performance are central to many theoretical accounts of cognitive processing. In recent years, several integrated performance measures have been proposed. A comparative study of the available measures [Vandierendonck, A. (2017). A comparison of methods to combine speed and accuracy measures of performance: A rejoinder on the binning procedure. Behavior Research Methods, 49, 653-673. DOI: https://doi.org/10.3758/s13428-016-0721-5] concluded that three of the measures, namely inverse efficiency score, rate correct score, and linear integrated speed-accuracy score achieved a balanced integration of speed and accuracy. As a follow-up on that study, these three measures were examined in data analyses from 13 (published and unpublished) experiments in the context of task switching. The correlations of the effect sizes in these integrated scores with the effect sizes obtained in latency and accuracy were high, but varied across the three integrated measures. The efficiency to detect effects supported by the speed and accuracy data was examined by means of signal detection analyses. The three measures efficiently detected effects present in either speed or accuracy, but the rate correct score was less efficient than the other two measures and it signalled a larger number of strong effects unsupported by the speed and accuracy data. It is concluded that while the rate correct score is better avoided, and the usage of the inverse efficiency score should be restricted to data with low overall error rates, the linear integrated speed-accuracy score proves to be valid.

SUBMITTER: Vandierendonck A

PROVIDER: S-EPMC6646946 | biostudies-literature | 2018 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Further Tests of the Utility of Integrated Speed-Accuracy Measures in Task Switching.

Vandierendonck André A

Journal of cognition 20180112 1

Speed and accuracy of performance are central to many theoretical accounts of cognitive processing. In recent years, several integrated performance measures have been proposed. A comparative study of the available measures [Vandierendonck, A. (2017). A comparison of methods to combine speed and accuracy measures of performance: A rejoinder on the binning procedure. <i>Behavior Research Methods, 49</i>, 653-673. DOI: https://doi.org/10.3758/s13428-016-0721-5] concluded that three of the measures, ...[more]

PMID: 31517182

Similar Datasets

Project description:Overall accuracy measures of medical tests are often used with unclear interpretations. To develop methods of calculating the overall accuracy of medical tests in the patient population. Algebraic equations based on Bayes' theorem. A new approach is proposed for calculating overall accuracy in the patient population. Examples and applications using published data are presented. The overall accuracy is the proportion of the correct test results. We introduce a clear distinction between the overall accuracy measures of medical tests that are aimed at the detection of a disease in a screening of populations for public health purposes in the general population and the overall accuracy measures of tests aimed at determining a diagnosis in individuals in a clinical setting. We show that the overall detection accuracy measure is obtained in a specific study that explores test accuracy among persons with known diagnoses and may be useful for public health screening tests. It is different from the overall diagnostic accuracy that could be calculated in the clinical setting for the evaluation of medical tests aimed at determining the individual patients' diagnoses. We show that the overall detection accuracy is constant and is not affected by the prevalence of the disease. In contrast, the overall diagnostic accuracy changes and is dependent on the prevalence. Moreover, it ranges according to the ratio between the sensitivity and specificity. Thus, when the sensitivity is greater than the specificity, the overall diagnostic accuracy increases with increasing prevalence, and vice versa, that is, when the sensitivity is lower than the specificity, the overall diagnostic accuracy decreases with increasing prevalence so that another test might be more useful for diagnostic procedures. Our paper suggests a new and more appropriate methodology for estimating the overall diagnostic accuracy of any medical test. This may be important for helping clinicians avoid errors.

Project description:IntroductionInterruptions in the emergency department (ED) are associated with clinical errors, yet are important when providing care to multiple patients. Screening triage electrocardiograms (ECG) for ST-segment elevation myocardial infarction (STEMI) represent a critical interrupting task that emergency physicians (EP) frequently encounter. To address interruptions such as ECG interpretation, many EPs engage in task switching, pausing their primary task to address an interrupting task. The impact of task switching on clinical errors in interpreting screening ECGs for STEMI remains unknown.MethodsResident and attending EPs were invited to participate in a crossover simulation trial. Physicians first completed a task-switching simulation in which they viewed patient presentations interrupted by clinical tasks, including screening ECGs requiring immediate interpretation before resuming the patient presentation. Participants then completed an uninterrupted simulation in which patient presentations and clinical tasks were completed sequentially without interruption. The primary outcome was accuracy of ECG interpretation for STEMI during task switching and uninterrupted simulations.ResultsThirty-five participants completed the study. We found no significant difference in accuracy of ECG interpretation for STEMI (task switching 0.89, uninterrupted 0.91, paired t-test p=0.21). Attending physician status (odds ratio [OR] [2.56], confidence interval [CI] [1.66-3.94], p<0.01) and inferior STEMI (OR [0.08], CI [0.04-0.14], p<0.01) were associated with increased and decreased odds of correct interpretation, respectively. Low self-reported confidence in interpretation was associated with decreased odds of correct interpretation in the task-switching simulation, but not in the uninterrupted simulation (interaction p=0.02).ConclusionIn our simulation, task switching was not associated with overall accuracy of ECG interpretation for STEMI. However, odds of correct interpretation decreased with inferior STEMI ECGs and when participants self-reported low confidence when interrupted. Our study highlights opportunities to improve through focused ECG training, as well as self-identification of "high-risk" screening ECGs prone to error during interrupted clinical workflow.

Dataset Information

Further Tests of the Utility of Integrated Speed-Accuracy Measures in Task Switching.

Publications

Further Tests of the Utility of Integrated Speed-Accuracy Measures in Task Switching.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets