Dataset Information

Comparing hormone therapy effects in two RCTs and two large observational studies that used similar methods for comprehensive data collection and outcome assessment.

ABSTRACT:

Objectives

Prospective observational studies (OSs) that collect adequate information about confounders can validly assess treatment consequences. However, what constitutes adequate information is unknown. This study investigated whether the extensive information collected by the Women's Health Initiative (WHI) in two OSs and two randomised controlled trials (RCTs) was adequate.

Design

Secondary analysis of WHI data. Cox regression was used to select from all baseline risk factors those that best predicted outcome. Cox regression that included these risk factors was used for two types of analyses: (1) comparing RCT and OS assessments of the effects of hormone therapy on outcome for participants with specific characteristics and (2) evaluating whether adjustment for measured confounders could eliminate outcome differences among datasets.

Setting

The WHI included more than 800 baseline risk factors and outcomes during a median follow-up of 8 years.

Participants

151 870 postmenopausal women ages 50-79.

Primary and secondary outcome measures

Myocardial infarction and stroke.

Results

RCT and OS results differed for the association of hormone therapy with outcome after adjusting for confounding factors and stratifying on factors that were hypothesised to modulate the effects of hormone therapy (eg, age and time since menopause) or that empirically modulated the effects of hormone therapy in this dataset (eg, blood pressure, previous coronary revascularisation and private medical insurance). Some of the four WHI datasets had significantly worse outcomes than others even after adjusting for risk and stratifying by type of hormone therapy, for example, the risk-adjusted HR for myocardial infarction was 1.37 (p<0.0001) in an RCT placebo group compared with an OS group not taking hormone therapy.

Conclusions

Apparently the WHI did not collect sufficient information to give reliable assessments of treatment effects. If the WHI did not collect sufficient data, it is likely that few OSs collect sufficient information.

SUBMITTER: Hartz A

PROVIDER: S-EPMC3717474 | biostudies-literature | 2013

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparing hormone therapy effects in two RCTs and two large observational studies that used similar methods for comprehensive data collection and outcome assessment.

Hartz Arthur A He Tao T Wallace Robert R Powers John J

BMJ open 20130715 7

<h4>Objectives</h4>Prospective observational studies (OSs) that collect adequate information about confounders can validly assess treatment consequences. However, what constitutes adequate information is unknown. This study investigated whether the extensive information collected by the Women's Health Initiative (WHI) in two OSs and two randomised controlled trials (RCTs) was adequate.<h4>Design</h4>Secondary analysis of WHI data. Cox regression was used to select from all baseline risk factors ...[more]

PMID: 23861441

Similar Datasets

Project description:Study questionWhich outcome measures are reported in RCTs for IVF?Summary answerMany combinations of numerator and denominator are in use, and are often employed in a manner that compromises the validity of the study.What is known alreadyThe choice of numerator and denominator governs the meaning, relevance and statistical integrity of a study's results. RCTs only provide reliable evidence when outcomes are assessed in the cohort of randomised participants, rather than in the subgroup of patients who completed treatment.Study design, size, durationReview of outcome measures reported in 142 IVF RCTs published in 2013 or 2014.Participants/materials, setting, methodsTrials were identified by searching the Cochrane Gynaecology and Fertility Specialised Register. English-language publications of RCTs reporting clinical or preclinical outcomes in peer-reviewed journals in the period 1 January 2013 to 31 December 2014 were eligible. Reported numerators and denominators were extracted. Where they were reported, we checked to see if live birth rates were calculated correctly using the entire randomised cohort or a later denominator.Main results and the role of chanceOver 800 combinations of numerator and denominator were identified (613 in no more than one study). No single outcome measure appeared in the majority of trials. Only 22 (43%) studies reporting live birth presented a calculation including all randomised participants or only excluding protocol violators. A variety of definitions were used for key clinical numerators: for example, a consensus regarding what should constitute an ongoing pregnancy does not appear to exist at present.Limitations, reasons for cautionSeveral of the included articles may have been secondary publications. Our categorisation scheme was essentially arbitrary, so the frequencies we present should be interpreted with this in mind. The analysis of live birth denominators was post hoc.Wider implications of the findingsThere is massive diversity in numerator and denominator selection in IVF trials due to its multistage nature, and this causes methodological frailty in the evidence base. The twin spectres of outcome reporting bias and analysis of non-randomised comparisons do not appear to be widely recognised. Initiatives to standardise outcome reporting, such as requiring all effectiveness studies to report live birth or cumulative live birth, are welcome. However, there is a need to recognise that early outcomes of treatment, such as stimulation response or embryo quality, may be appropriate choices of primary outcome for early phase studies.Study funding/competing interestsJ.W. is funded by a Doctoral Research Fellowship from the National Institute for Health Research. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health. J.W. also declares that publishing research is beneficial to his career. J.W. and A.V. are statistical editors, and M.S. is Information Specialist, for the Cochrane Gynaecology and Fertility Group, although the views expressed here are not necessarily those of the group. D.R.B. is funded by the NHS as Scientific Director of a clinical IVF service. The authors declare no other conflicts of interest.

Dataset Information

Comparing hormone therapy effects in two RCTs and two large observational studies that used similar methods for comprehensive data collection and outcome assessment.

Objectives

Design

Setting

Participants

Primary and secondary outcome measures

Results

Conclusions

Publications

Comparing hormone therapy effects in two RCTs and two large observational studies that used similar methods for comprehensive data collection and outcome assessment.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure