Dataset Information

Understanding test accuracy research: a test consequence graphic.

ABSTRACT:

Background

Presenting results of diagnostic test accuracy research so that it is accessible to users is challenging. Commonly used accuracy measures (e.g. sensitivity and specificity) are poorly understood by health professionals and the public. Evidence suggests that presenting probabilities as natural frequencies rather than percentages facilitates understanding. We present a test consequence graphic to display results based on natural frequencies and test consequences.

Methods

The graphic was developed as part of a project to develop guidance for writing plain language summaries for Cochrane diagnostic test accuracy (DTA) reviews. Using a mixed methods approach (focus groups, user testing, web-based surveys, public engagement and piloting), the graphic emerged as a clear preference out of a range of methods for presenting probabilistic information (text only, numbers embedded in text, annotated graphic) across patient representatives, media representatives and health professionals. The structure of the graphic was refined during the research process.

Results

The test consequence graphic displays the results of diagnostic test accuracy study or review as natural frequencies based on a hypothetical cohort of 1000 patients receiving the test.

Conclusions

The test consequence graphic provides a tool to help researchers communicate the results of diagnostic research in a simple, easy to access format and encourage meaningful application of research findings to practice. Key to this is linking estimates of test accuracy to potential downstream consequences of testing.

SUBMITTER: Whiting P

PROVIDER: S-EPMC6460548 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Understanding test accuracy research: a test consequence graphic.

Whiting Penny P Davenport Clare C

Diagnostic and prognostic research 20180201

<h4>Background</h4>Presenting results of diagnostic test accuracy research so that it is accessible to users is challenging. Commonly used accuracy measures (e.g. sensitivity and specificity) are poorly understood by health professionals and the public. Evidence suggests that presenting probabilities as natural frequencies rather than percentages facilitates understanding. We present a test consequence graphic to display results based on natural frequencies and test consequences.<h4>Methods</h4> ...[more]

PMID: 31093552

Similar Datasets

Project description:ImportanceDespite acknowledging that medical jargon should be avoided, health care practitioners frequently use it when communicating with patients.ObjectiveTo characterize the understanding of common medical jargon terms by surveying a cross section of the general public and studying phrases that have established meanings in regular usage but different meanings in a medical context (eg, negative and positive test results).Design, setting, and participantsIn this cross-sectional study, participants indicated their understanding of phrases that may have different meanings in medicine than in colloquial English via a mix of short answer and multiple choice questions. Several questions included paired phrases to assess for differences in understanding with or without jargon. Volunteers were recruited at the 2021 Minnesota State Fair near St Paul, Minnesota. An electronic survey was given to a volunteer sample of 215 adults (>18 years) who did not work or train to work in the medical field and spoke and read English.ExposuresCompleting a written or verbal survey.Main outcomes and measuresThe main outcome was an accurate understanding of the medical terminology. Free-text responses were coded by 2 researchers for comprehension. Secondary outcomes looked for associations between volunteer demographics and understanding.ResultsThe 215 respondents (135 [63%] female; mean [SD] age, 42 [17] years) demonstrated a varied ability to interpret medical jargon phrases. For example, most participants (207 [96%]) knew that negative cancer screening results meant they did not have cancer, but fewer participants (143 [79%]) knew that the phrase "your tumor is progressing" was bad news, or that positive lymph nodes meant the cancer had spread (170 [67%]). While most (171 [80%]) recognized that an unremarkable chest radiography was good news, only 44 participants (21%) correctly understood that a clinician saying their radiography was impressive was generally bad news. In each of the paired phrases comparing jargon vs nonjargon approaches, the nonjargon phrase was understood significantly better (P < .001).Conclusions and relevanceThese findings suggest that several common phrases are misunderstood when used in a medical setting, with the interpreted meaning frequently the exact opposite of what is intended.

Project description:OBJECTIVE:To assess the accuracy of the AbC-19 Rapid Test lateral flow immunoassay for the detection of previous severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. DESIGN:Test accuracy study. SETTING:Laboratory based evaluation. PARTICIPANTS:2847 key workers (healthcare staff, fire and rescue officers, and police officers) in England in June 2020 (268 with a previous polymerase chain reaction (PCR) positive result (median 63 days previously), 2579 with unknown previous infection status); and 1995 pre-pandemic blood donors. MAIN OUTCOME MEASURES:AbC-19 sensitivity and specificity, estimated using known negative (pre-pandemic) and known positive (PCR confirmed) samples as reference standards and secondly using the Roche Elecsys anti-nucleoprotein assay, a highly sensitive laboratory immunoassay, as a reference standard in samples from key workers. RESULTS:Test result bands were often weak, with positive/negative discordance by three trained laboratory staff for 3.9% of devices. Using consensus readings, for known positive and negative samples sensitivity was 92.5% (95% confidence interval 88.8% to 95.1%) and specificity was 97.9% (97.2% to 98.4%). Using an immunoassay reference standard, sensitivity was 94.2% (90.7% to 96.5%) among PCR confirmed cases but 84.7% (80.6% to 88.1%) among other people with antibodies. This is consistent with AbC-19 being more sensitive when antibody concentrations are higher, as people with PCR confirmation tended to have more severe disease whereas only 62% (218/354) of seropositive participants had had symptoms. If 1 million key workers were tested with AbC-19 and 10% had actually been previously infected, 84?700 true positive and 18?900 false positive results would be projected. The probability that a positive result was correct would be 81.7% (76.8% to 85.8%). CONCLUSIONS:AbC-19 sensitivity was lower among unselected populations than among PCR confirmed cases of SARS-CoV-2, highlighting the scope for overestimation of assay performance in studies involving only PCR confirmed cases, owing to "spectrum bias." Assuming that 10% of the tested population have had SARS-CoV-2 infection, around one in five key workers testing positive with AbC-19 would be false positives. STUDY REGISTRATION:ISRCTN 56609224.

Dataset Information

Understanding test accuracy research: a test consequence graphic.

Background

Methods

Results

Conclusions

Publications

Understanding test accuracy research: a test consequence graphic.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets