Dataset Information

Meta-analysis of test accuracy studies using imputation for partial reporting of multiple thresholds.

ABSTRACT:

Introduction

For tests reporting continuous results, primary studies usually provide test performance at multiple but often different thresholds. This creates missing data when performing a meta-analysis at each threshold. A standard meta-analysis (no imputation [NI]) ignores such missing data. A single imputation (SI) approach was recently proposed to recover missing threshold results. Here, we propose a new method that performs multiple imputation of the missing threshold results using discrete combinations (MIDC).

Methods

The new MIDC method imputes missing threshold results by randomly selecting from the set of all possible discrete combinations which lie between the results for 2 known bounding thresholds. Imputed and observed results are then synthesised at each threshold. This is repeated multiple times, and the multiple pooled results at each threshold are combined using Rubin's rules to give final estimates. We compared the NI, SI, and MIDC approaches via simulation.

Results

Both imputation methods outperform the NI method in simulations. There was generally little difference in the SI and MIDC methods, but the latter was noticeably better in terms of estimating the between-study variances and generally gave better coverage, due to slightly larger standard errors of pooled estimates. Given selective reporting of thresholds, the imputation methods also reduced bias in the summary receiver operating characteristic curve. Simulations demonstrate the imputation methods rely on an equal threshold spacing assumption. A real example is presented.

Conclusions

The SI and, in particular, MIDC methods can be used to examine the impact of missing threshold results in meta-analysis of test accuracy studies.

SUBMITTER: Ensor J

PROVIDER: S-EPMC5873416 | biostudies-literature | 2018 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Meta-analysis of test accuracy studies using imputation for partial reporting of multiple thresholds.

Ensor J J Deeks J J JJ Martin E C EC Riley R D RD

Research synthesis methods 20171122 1

<h4>Introduction</h4>For tests reporting continuous results, primary studies usually provide test performance at multiple but often different thresholds. This creates missing data when performing a meta-analysis at each threshold. A standard meta-analysis (no imputation [NI]) ignores such missing data. A single imputation (SI) approach was recently proposed to recover missing threshold results. Here, we propose a new method that performs multiple imputation of the missing threshold results using ...[more]

PMID: 29052347

Similar Datasets

Project description:Primary studies examining the accuracy of a continuous test evaluate its sensitivity and specificity at one or more thresholds. Meta-analysts then usually perform a separate meta-analysis for each threshold. However, the number of studies available for each threshold is often very different, as primary studies are inconsistent in the thresholds reported. Furthermore, of concern is selective reporting bias, because primary studies may be less likely to report a threshold when it gives low sensitivity and/or specificity estimates. This may lead to biased meta-analysis results. We developed an exploratory method to examine the potential impact of missing thresholds on conclusions from a test accuracy meta-analysis.Our method identifies studies that contain missing thresholds bounded between a pair of higher and lower thresholds for which results are available. The bounded missing threshold results (two-by-two tables) are then imputed, by assuming a linear relationship between threshold value and each of logit-sensitivity and logit-specificity. The imputed results are then added to the meta-analysis, to ascertain if original conclusions are robust. The method is evaluated through simulation, and application made to 13 studies evaluating protein:creatinine ratio (PCR) for detecting proteinuria in pregnancy with 23 different thresholds, ranging from one to seven per study.The simulation shows the imputation method leads to meta-analysis estimates with smaller mean-square error. In the PCR application, it provides 50 additional results for meta-analysis and their inclusion produces lower test accuracy results than originally identified. For example, at a PCR threshold of 0.16, the summary specificity is 0.80 when using the original data, but 0.66 when also including the imputed data. At a PCR threshold of 0.25, the summary sensitivity is reduced from 0.95 to 0.85 when additionally including the imputed data.The imputation method is a practical tool for researchers (often non-statisticians) to explore the potential impact of missing threshold results on their meta-analysis conclusions. Software is available to implement the method. In the PCR example, it revealed threshold results are vulnerable to the missing data, and so stimulates the need for advanced statistical models or, preferably, individual patient data from primary studies.

Project description:AimPartial-mouth recording protocols often result in underestimation of population prevalence and extent of periodontitis. We posit that multiple imputation of measures such as clinical attachment loss for nonselected tooth sites in partial-mouth samples can reduce bias in periodontitis estimates.MethodsMultiple imputation for correlated site-level dichotomous outcomes in a generalized estimating equations framework is used to impute site-level binary indicators for clinical attachment loss exceeding a fixed threshold in partial-mouth samples. Periodontitis case definitions are applied to the imputed "complete" dentitions, enabling estimation of prevalence and other summaries of periodontitis for partial-mouth samples as if for full-mouth examinations. A multiple imputation-bootstrap procedure is described and applied for point and variance estimation of these periodontitis measures. The procedure is evaluated with pseudo-partial-mouth samples based on random site selection protocols of 28 to 84 periodontal sites repeatedly generated from full-mouth periodontal examinations of 3,621 participants in the 2013 to 2014 National Health and Nutrition Examination Survey (NHANES) survey.ResultsMultiple imputation applied to partial-mouth samples overestimated periodontitis mean extent, defined as the number of sites with clinical attachment loss 3 mm or greater, by 9.5% in random site selection protocols with 84 sites and overestimated prevalence by 5% to 10% in all the evaluated protocols.ConclusionsIn the 2013 to 2014 NHANES data, multiple imputation of site-level periodontal indicators provides less biased estimates of periodontitis prevalence and extent than has been reported from estimates based on the direct application of full-mouth case definitions to partial-mouth samples. Multiple imputation provides a promising solution to the longstanding, vexing problem of estimation bias in partial-mouth recording, with potential application to a wide array of case definitions, periodontitis measures, and partial recording protocols.Knowledge transfer statementPartial-mouth sampling, while a resource-efficient strategy for obtaining oral disease estimates, often results in underestimation of periodontitis metrics. Multiple imputation for nonselected periodontal sites produces pseudo-full-mouth data sets that may be analyzed and combined to produce estimates with small bias.

Project description:IntroductionStandards for Reporting of Diagnostic Accuracy Study (STARD) was developed to improve the completeness and transparency of reporting in studies investigating diagnostic test accuracy. However, its current form, STARD 2015 does not address the issues and challenges raised by artificial intelligence (AI)-centred interventions. As such, we propose an AI-specific version of the STARD checklist (STARD-AI), which focuses on the reporting of AI diagnostic test accuracy studies. This paper describes the methods that will be used to develop STARD-AI.Methods and analysisThe development of the STARD-AI checklist can be distilled into six stages. (1) A project organisation phase has been undertaken, during which a Project Team and a Steering Committee were established; (2) An item generation process has been completed following a literature review, a patient and public involvement and engagement exercise and an online scoping survey of international experts; (3) A three-round modified Delphi consensus methodology is underway, which will culminate in a teleconference consensus meeting of experts; (4) Thereafter, the Project Team will draft the initial STARD-AI checklist and the accompanying documents; (5) A piloting phase among expert users will be undertaken to identify items which are either unclear or missing. This process, consisting of surveys and semistructured interviews, will contribute towards the explanation and elaboration document and (6) On finalisation of the manuscripts, the group's efforts turn towards an organised dissemination and implementation strategy to maximise end-user adoption.Ethics and disseminationEthical approval has been granted by the Joint Research Compliance Office at Imperial College London (reference number: 19IC5679). A dissemination strategy will be aimed towards five groups of stakeholders: (1) academia, (2) policy, (3) guidelines and regulation, (4) industry and (5) public and non-specific stakeholders. We anticipate that dissemination will take place in Q3 of 2021.

Dataset Information

Meta-analysis of test accuracy studies using imputation for partial reporting of multiple thresholds.

Introduction

Methods

Results

Conclusions

Publications

Meta-analysis of test accuracy studies using imputation for partial reporting of multiple thresholds.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets