Dataset Information

The Missing Medians: Exclusion of Ordinal Data from Meta-Analyses.

ABSTRACT:

Background

Meta-analyses are considered the gold standard of evidence-based health care, and are used to guide clinical decisions and health policy. A major limitation of current meta-analysis techniques is their inability to pool ordinal data. Our objectives were to determine the extent of this problem in the context of neurological rating scales and to provide a solution.

Methods

Using an existing database of clinical trials of oral neuroprotective therapies, we identified the 6 most commonly used clinical rating scales and recorded how data from these scales were reported and analysed. We then identified systematic reviews of studies that used these scales (via the Cochrane database) and recorded the meta-analytic techniques used. Finally, we identified a statistical technique for calculating a common language effect size measure for ordinal data.

Results

We identified 103 studies, with 128 instances of the 6 clinical scales being reported. The majority- 80%-reported means alone for central tendency, with only 13% reporting medians. In analysis, 40% of studies used parametric statistics alone, 34% of studies employed non-parametric analysis, and 26% did not include or specify analysis. Of the 60 systematic reviews identified that included meta-analysis, 88% used mean difference and 22% employed difference in proportions; none included rank-based analysis. We propose the use of a rank-based generalised odds ratio (WMW GenOR) as an assumption-free effect size measure that is easy to compute and can be readily combined in meta-analysis.

Conclusion

There is wide scope for improvement in the reporting and analysis of ordinal data in the literature. We hope that adoption of the WMW GenOR will have the dual effect of improving the reporting of data in individual studies while also increasing the inclusivity (and therefore validity) of meta-analyses.

SUBMITTER: Cumming TB

PROVIDER: S-EPMC4689383 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The Missing Medians: Exclusion of Ordinal Data from Meta-Analyses.

Cumming Toby B TB Churilov Leonid L Sena Emily S ES

PloS one 20151223 12

<h4>Background</h4>Meta-analyses are considered the gold standard of evidence-based health care, and are used to guide clinical decisions and health policy. A major limitation of current meta-analysis techniques is their inability to pool ordinal data. Our objectives were to determine the extent of this problem in the context of neurological rating scales and to provide a solution.<h4>Methods</h4>Using an existing database of clinical trials of oral neuroprotective therapies, we identified the 6 ...[more]

PMID: 26697876

Similar Datasets

Project description:BackgroundMissing participant outcome data (MOD) are ubiquitous in systematic reviews with network meta-analysis (NMA) as they invade from the inclusion of clinical trials with reported participant losses. There are available strategies to address aggregate MOD, and in particular binary MOD, while considering the missing at random (MAR) assumption as a starting point. Little is known about their performance though regarding the meta-analytic parameters of a random-effects model for aggregate binary outcome data as obtained from trial-reports (i.e. the number of events and number of MOD out of the total randomised per arm).MethodsWe used four strategies to handle binary MOD under MAR and we classified these strategies to those modelling versus excluding/imputing MOD and to those accounting for versus ignoring uncertainty about MAR. We investigated the performance of these strategies in terms of core NMA estimates by performing both an empirical and simulation study using random-effects NMA based on electrical network theory. We used Bland-Altman plots to illustrate the agreement between the compared strategies, and we considered the mean bias, coverage probability and width of the confidence interval to be the frequentist measures of performance.ResultsModelling MOD under MAR agreed with exclusion and imputation under MAR in terms of estimated log odds ratios and inconsistency factor, whereas accountability or not of the uncertainty regarding MOD affected intervention hierarchy and precision around the NMA estimates: strategies that ignore uncertainty about MOD led to more precise NMA estimates, and increased between-trial variance. All strategies showed good performance for low MOD (<5%), consistent evidence and low between-trial variance, whereas performance was compromised for large informative MOD (>?20%), inconsistent evidence and substantial between-trial variance, especially for strategies that ignore uncertainty due to MOD.ConclusionsThe analysts should avoid applying strategies that manipulate MOD before analysis (i.e. exclusion and imputation) as they implicate the inferences negatively. Modelling MOD, on the other hand, via a pattern-mixture model to propagate the uncertainty about MAR assumption constitutes both conceptually and statistically proper strategy to address MOD in a systematic review.

Project description:To compare the characteristics, quality and treatment effects of randomised clinical trials (RCTs) by individual patient data (IPD) availability, in trials eligible for 18 IPD meta-analyses (MA).Trial characteristics, risk of bias (RoB) and hazard ratio (HR) for overall survival were extracted from IPD-MA publications and/or RCTs publications. Data for the RoB assessment were extracted for a subset of 73 RCTs. Two investigators blinded to whether IPD was available or not evaluated the RoB for these trials. Treatment effects were compared using ratios of global HRs (RHRs) of IPD-unavailable trials and IPD-available trials. RHR were pooled using a fixed-effect model.We examined the IPD availability for each trial eligible for each IPD-MA; when the IPD was not available for a trial, we used information from published sources.We selected all published IPD-MAs conducted at Gustave Roussy and the RCTs eligible for each.349 RCTs (73?018 patients) from 18 MAs were eligible: 60 RCTs (5890 patients) had unavailable IPD and 289 RCTs (67?128 patients) had available IPD. The main reason for IPD unavailability was data loss by investigators. IPD-unavailable trials were smaller (p<0.001), more often monocentric (p<0.001) and non-international (p=0.0004) than IPD-available trials. Geographical areas differed (p=0.054) between IPD-unavailable IPD-available trials. RoB was higher in IPD-unavailable RCTs for random sequence generation (p=0.007) and allocation concealment (p=0.006). The HR and 95% confidence interval (CI) for overall survival were extractable from publications in 23/60 IPD-unavailable trials included in 10 different MAs. Treatment effects were significantly greater for IPD-unavailable trials compared with IPD-available trials (RHR=0.86 (95% CI 0.75 to 0.98)).IPD-unavailable RCTs were significantly different from IPD-available RCTs in terms of trial characteristics and were at greater RoB. IPD-unavailable RCTs had a significantly greater treatment effect.

Project description:BackgroundMissing data are an inevitable challenge in Randomised Controlled Trials (RCTs), particularly those with Patient Reported Outcome Measures. Methodological guidance suggests that to avoid incorrect conclusions, studies should undertake sensitivity analyses which recognise that data may be 'missing not at random' (MNAR). A recommended approach is to elicit expert opinion about the likely outcome differences for those with missing versus observed data. However, few published trials plan and undertake these elicitation exercises, and so lack the external information required for these sensitivity analyses. The aim of this paper is to provide a framework that anticipates and allows for MNAR data in the design and analysis of clinical trials.MethodsWe developed a framework for performing and using expert elicitation to frame sensitivity analysis in RCTs with missing outcome data. The framework includes the following steps: first defining the scope of the elicitation exercise, second developing the elicitation tool, third eliciting expert opinion about the missing outcomes, fourth evaluating the elicitation results, and fifth analysing the trial data. We provide guidance on key practical challenges that arise when adopting this approach in trials: the criteria for identifying relevant experts, the outcome scale for presenting data to experts, the appropriate representation of expert opinion, and the evaluation of the elicitation results.The framework was developed within the POPPI trial, which investigated whether a preventive, complex psychological intervention, commenced early in ICU, would reduce the development of patient-reported post-traumatic stress disorder symptom severity, and improve health-related quality of life. We illustrate the key aspects of the proposed framework using the POPPI trial.ResultsFor the POPPI trial, 113 experts were identified with potentially suitable knowledge and asked to participate in the elicitation exercise. The 113 experts provided 59 usable elicitation questionnaires. The sensitivity analysis found that the results from the primary analysis were robust to alternative MNAR mechanisms.ConclusionsFuture studies can adopt this framework to embed expert elicitation within the design of clinical trials. This will provide the information required for MNAR sensitivity analyses that examine the robustness of the trial conclusions to alternative, but realistic assumptions about the missing data.

Dataset Information

The Missing Medians: Exclusion of Ordinal Data from Meta-Analyses.

Background

Methods

Results

Conclusion

Publications

The Missing Medians: Exclusion of Ordinal Data from Meta-Analyses.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets