Dataset Information

Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey.

ABSTRACT:

Background

Decisionmakers and guideline developers demand rapid syntheses of the evidence when time sensitive evidence-informed decisions are required. A potential trade-off of such rapid reviews is that their results can have less reliability than results of systematic reviews that can lead to an increased risk of making incorrect decisions or recommendations. We sought to determine how much incremental uncertainty about the correctness of an answer guideline developers and health policy decisionmakers are willing to accept in exchange for a rapid evidence-synthesis.

Methods

Employing a purposive sample, we conducted an international web-based, anonymous survey of decisionmakers and guideline developers. Based on a clinical treatment, a public health, and a clinical prevention scenario, participants indicated the maximum risk of getting an incorrect answer from a rapid review that they would be willing to accept. We carefully reviewed data and performed descriptive statistical analyses.

Results

In total, 325 (58.5%) of 556 participants completed our survey and were eligible for analysis. The median acceptable incremental risk for getting an incorrect answer from a rapid review across all three scenarios was 10.0% (interquartile range [IQR] 5.0-15.0). Acceptable risks were similar for the clinical treatment (n = 313, median 10.0% [IQR 5.0-15.0]) and the public health scenarios (n = 320, median 10.0% [IQR 5.0-15.0]) and lower for the clinical prevention scenario (n = 312, median 6.5% [IQR 5.0-10.5]).

Conclusions

Findings suggest that decisionmakers are willing to accept some trade-off in validity in exchange for a rapid review. Nevertheless, they expect the validity of rapid reviews to come close to that of systematic reviews.

SUBMITTER: Wagner G

PROVIDER: S-EPMC5557322 | biostudies-literature | 2017 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey.

Wagner Gernot G Nussbaumer-Streit Barbara B Greimel Judith J Ciapponi Agustín A Gartlehner Gerald G

BMC medical research methodology 20170814 1

<h4>Background</h4>Decisionmakers and guideline developers demand rapid syntheses of the evidence when time sensitive evidence-informed decisions are required. A potential trade-off of such rapid reviews is that their results can have less reliability than results of systematic reviews that can lead to an increased risk of making incorrect decisions or recommendations. We sought to determine how much incremental uncertainty about the correctness of an answer guideline developers and health polic ...[more]

PMID: 28806999

Similar Datasets

Project description:Intolerance of Uncertainty is a trans-diagnostic process that spans a range of emotional disorders and it is usually measured through the Intolerance of Uncertainty Scale-12. The current study aims at investigating some issues in the assessment of Intolerance of Uncertainty (IU) through the Italian Intolerance of Uncertainty Scale-Revised, a measure adapted from the Intolerance of Uncertainty Scale-12 to assess IU across the lifespan. In particular we address the factor structure among a large community sample, measurement invariance across gender, age, and over time, together with reliability and validity of the overall scale and its subscales. The questionnaire was administered to community (N = 761; mean age = 35.86 ± 14.01 years) and undergraduate (N = 163; mean age = 21.16 ± 2.64 years) participants, together with other self-report measures assessing constructs theoretically related to IU. The application of a bifactor model shows that the Italian Intolerance of Uncertainty Scale-Revised possesses a robust general factor, thus supporting the use of the unit-weighted total score of the questionnaire as a measure of the construct. Furthermore, measurement invariance across gender, age, and over time is supported. Finally, the Italian Intolerance of Uncertainty Scale-Revised appears to possess adequate reliability and validity. These findings support the unidimensionality of the measure, a conceptually reasonable result in line with the trans-diagnostic nature of Intolerance of Uncertainty. In addition, this study and comparison with published factor structures of the Intolerance of Uncertainty Scale-12 and of the Intolerance of Uncertainty Scale-Revised identify some issues for the internal structure of the measure. In particular, concern is expressed for the Prospective IU subscale. In light of the promising psychometric properties, the use of the Italian Intolerance of Uncertainty Scale-Revised as a univocal measure is encouraged in both research and clinical practice.

Project description:BackgroundA variety of systems have been developed to grade evidence and develop recommendations based on the available evidence. However, development of guidelines for medical tests is especially challenging given the typical indirectness of the evidence; direct evidence of the effects of testing on patient important outcomes is usually absent. We compared grading systems for medical tests on how they use evidence in guideline development.MethodsWe used a systematic strategy to look for grading systems specific to medical tests in PubMed, professional guideline websites, via personal correspondence, and handsearching back references of key articles. Using the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument as a starting point, we defined two sets of characteristics to describe these systems: methodological and process ones. Methodological characteristics are features relating to how evidence is gathered, appraised, and used in recommendations. Process characteristics are those relating to the guideline development process. Data were extracted in duplicate and differences resolved through discussion.ResultsTwelve grading systems could be included. All varied in the degree to which methodological and process characteristics were addressed. Having a clinical scenario, identifying the care pathway and/or developing an analytical framework, having explicit criteria for appraising and linking indirect evidence, and having explicit methodologies for translating evidence into recommendations were least frequently addressed. Five systems at most addressed these, to varying degrees of explicitness and completeness. Process wise, features most frequently addressed included involvement of relevant professional groups (8/12), external peer review of completed guidelines (9/12), and recommendations on methods for dissemination (8/12). Characteristics least often addressed were whether the system was piloted (3/12) and funder information (3/12).ConclusionsFive systems for grading evidence about medical tests in guideline development addressed to differing degrees of explicitness the need for and appraisal of different bodies of evidence, the linking of such evidence, and its translation into recommendations. At present, no one system addressed the full complexity of gathering, assessing and linking different bodies of evidence.

Project description:Wildlife diseases have important implications for wildlife and human health, the preservation of biodiversity and the resilience of ecosystems. However, understanding disease dynamics and the impacts of pathogens in wild populations is challenging because these complex systems can rarely, if ever, be observed without error. Uncertainty in disease ecology studies is commonly defined in terms of either heterogeneity in detectability (due to variation in the probability of encountering, capturing, or detecting individuals in their natural habitat) or uncertainty in disease state assignment (due to misclassification errors or incomplete information). In reality, however, uncertainty in disease ecology studies extends beyond these components of observation error and can arise from multiple varied processes, each of which can lead to bias and a lack of precision in parameter estimates. Here, we present an inventory of the sources of potential uncertainty in studies that attempt to quantify disease-relevant parameters from wild populations (e.g., prevalence, incidence, transmission rates, force of infection, risk of infection, persistence times, and disease-induced impacts). We show that uncertainty can arise via processes pertaining to aspects of the disease system, the study design, the methods used to study the system, and the state of knowledge of the system, and that uncertainties generated via one process can propagate through to others because of interactions between the numerous biological, methodological and environmental factors at play. We show that many of these sources of uncertainty may not be immediately apparent to researchers (for example, unidentified crypticity among vectors, hosts or pathogens, a mismatch between the temporal scale of sampling and disease dynamics, demographic or social misclassification), and thus have received comparatively little consideration in the literature to date. Finally, we discuss the type of bias or imprecision introduced by these varied sources of uncertainty and briefly present appropriate sampling and analytical methods to account for, or minimise, their influence on estimates of disease-relevant parameters. This review should assist researchers and practitioners to navigate the pitfalls of uncertainty in wildlife disease ecology studies.

Dataset Information

Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey.

Background

Methods

Results

Conclusions

Publications

Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets