Dataset Information

Current use of routinely collected health data to complement randomized controlled trials: a meta-epidemiological survey.

ABSTRACT:

Background

Studies that use routinely collected health data (RCD studies) are advocated to complement evidence from randomized controlled trials (RCTs) for comparative effectiveness research and to inform health care decisions when RCTs would be unfeasible. We aimed to evaluate the current use of routinely collected health data to complement RCT evidence.

Methods

We searched PubMed for RCD studies published to 2010 that evaluated the comparative effectiveness of medical treatments on mortality using propensity scores. We identified RCTs of the same treatment comparisons and evaluated how frequently the RCD studies analyzed treatments that had not been compared previously in randomized trials. When RCTs did exist, we noted the claimed motivations for each RCD study. We also analyzed the citation impact of the RCD studies.

Results

Of 337 eligible RCD studies identified, 231 (68.5%) analyzed treatments that had already been compared in RCTs. The study investigators rarely claimed that it would be unethical (6/337) or difficult (18/337) to perform RCTs on the same question. Evidence from RCTs was mentioned or cited by authors of 213 RCD studies. The most common motivations for conducting the RCD studies were alleged limited generalizability of trial results to the "real world" (37.6%), evaluation of specific outcomes (31.9%) or specific populations (23.5%), and inconclusive or inconsistent evidence from randomized trials (25.8%). Studies evaluating "real world" effects had the lowest citation impact.

Interpretation

Most of the RCD studies we identified explored comparative treatment effects that had already been investigated in RCTs. The objective of such studies needs to shift more toward answering pivotal questions that are not supported by trial evidence or for which RCTs would be unfeasible.

SUBMITTER: Hemkens LG

PROVIDER: S-EPMC4933635 | biostudies-literature | 2016 Apr-Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Current use of routinely collected health data to complement randomized controlled trials: a meta-epidemiological survey.

Hemkens Lars G LG Contopoulos-Ioannidis Despina G DG Ioannidis John P A JP

CMAJ open 20160406 2

<h4>Background</h4>Studies that use routinely collected health data (RCD studies) are advocated to complement evidence from randomized controlled trials (RCTs) for comparative effectiveness research and to inform health care decisions when RCTs would be unfeasible. We aimed to evaluate the current use of routinely collected health data to complement RCT evidence.<h4>Methods</h4>We searched PubMed for RCD studies published to 2010 that evaluated the comparative effectiveness of medical treatments ...[more]

PMID: 27398355

Similar Datasets

Project description:ObjectiveTo assess differences in estimated treatment effects for mortality between observational studies with routinely collected health data (RCD; that are published before trials are available) and subsequent evidence from randomized controlled trials on the same clinical question.DesignMeta-epidemiological survey.Data sourcesPubMed searched up to November 2014.MethodsEligible RCD studies were published up to 2010 that used propensity scores to address confounding bias and reported comparative effects of interventions for mortality. The analysis included only RCD studies conducted before any trial was published on the same topic. The direction of treatment effects, confidence intervals, and effect sizes (odds ratios) were compared between RCD studies and randomized controlled trials. The relative odds ratio (that is, the summary odds ratio of trial(s) divided by the RCD study estimate) and the summary relative odds ratio were calculated across all pairs of RCD studies and trials. A summary relative odds ratio greater than one indicates that RCD studies gave more favorable mortality results.ResultsThe evaluation included 16 eligible RCD studies, and 36 subsequent published randomized controlled trials investigating the same clinical questions (with 17,275 patients and 835 deaths). Trials were published a median of three years after the corresponding RCD study. For five (31%) of the 16 clinical questions, the direction of treatment effects differed between RCD studies and trials. Confidence intervals in nine (56%) RCD studies did not include the RCT effect estimate. Overall, RCD studies showed significantly more favorable mortality estimates by 31% than subsequent trials (summary relative odds ratio 1.31 (95% confidence interval 1.03 to 1.65; I(2)=0%)).ConclusionsStudies of routinely collected health data could give different answers from subsequent randomized controlled trials on the same clinical questions, and may substantially overestimate treatment effects. Caution is needed to prevent misguided clinical decision making.

Project description:BackgroundLittle is known about Canadians' knowledge of and level of support for using administrative and other large, routinely collected data for health research, despite the benefits of this type of research to patients, health care systems and society. We sought to benchmark the views of Canadian adults on this topic.MethodsResearchers and patient leaders of 3 joint and skin disease organizations codeveloped a cross-sectional online survey that was conducted between January and August 2017. The patient partners were engaged as full partners. Recruitment was mainly through the organizations' websites, email and social media. The survey captured respondents' initial perceptions, then (after background information on the topic was provided) elicited their views on the benefits of health research using routinely collected data, data access/privacy concerns, ongoing perceptions and educational needs.ResultsOf the 230 people who consented, 183 (79.6%) started the survey, and 151 (65.6%) completed the survey. Of the 151, 117 (77.5%) were women, 84 (55.6%) were British Columbians, 87 (57.6%) were university graduates, and 101 (66.9%) had a chronic disease. At the beginning of the survey, 119 respondents (78.8%) felt positively about the use of routinely collected data for health research. Respondents identified the ability to study long-term treatment effects and rare events (114 [75.5%]) and large numbers of people (110 [72.8%]) as key benefits. Deidentification of personal information was the top privacy measure (135 [89.4%]), and 101 respondents (66.9%) wanted to learn more about data stewards' granting access to data. On survey completion, more respondents (141 [93.4%]) felt positively about the use of routinely collected data, but only 87 (57.6%) were confident about data security and privacy.InterpretationRespondents generally supported the use of deidentified routinely collected data for health research. Although further investigation is needed with more representative samples, our findings suggest that additional education, especially about access and privacy controls, may enhance public support for research endeavours using these data.

Project description:BackgroundA promising approach to reduce the increasing costs of clinical trials is the use of routinely collected health data as participant data. However, the quality of this data could limit its usability as trial participant data.MethodsThe BOSS trial is a randomised controlled trial comparing regular endoscopies versus endoscopies at need in patients with Barrett's oesophagus with primary endpoint death. Data on death and cancer collected every 2 years after randomisation (trial-specific data) were compared to data received annually (all patients on one date) from the routinely collected health data source National Health Service (NHS) Digital. We investigated completeness, agreement and timeliness and looked at the implications for the primary trial outcome. Completeness and agreement were assessed by evaluating the number of reported and missing cases and any disparities between reported dates. Timeliness was considered by graphing the year a death was first reported in the trial-specific data against that for NHS Digital data. Implications on the primary trial outcome, overall survival, of using one of the data sources alone were investigated using Kaplan-Meier graphs. To assess the utility of cause of death and cancer diagnoses, oesophageal cancer cases were compared.ResultsNHS Digital datasets included more deaths and often reported them sooner than the trial-specific data. The number reported as being from oesophageal cancer was similar in both datasets. Due to time lag in reporting and missing cases, the event rate appeared higher using the NHS Digital data.ConclusionNHS Digital death data is useful for calculating overall survival where trial-specific follow-up is only every 2 years from randomisation and the follow-up requires patient response. The cancer data was not a large enough sample to assess usability. We suggest that this assessment of registry data is done for more phase III RCTs and for more registry data to get a more complete picture of when RCHD would be useful in phase III RCT.Trial registrationISRCTN54190466 (BOSS) 1 Oct 2009.

Dataset Information

Current use of routinely collected health data to complement randomized controlled trials: a meta-epidemiological survey.

Background

Methods

Results

Interpretation

Publications

Current use of routinely collected health data to complement randomized controlled trials: a meta-epidemiological survey.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets