Dataset Information

Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty.

ABSTRACT: In the presence of time-dependent confounding, there are several methods available to estimate treatment effects. With correctly specified models and appropriate structural assumptions, any of these methods could provide consistent effect estimates, but with real-world data, all models will be misspecified and it is difficult to know if assumptions are violated. In this paper, we investigate five methods: inverse probability weighting of marginal structural models, history-adjusted marginal structural models, sequential conditional mean models, g-computation formula, and g-estimation of structural nested models. This work is motivated by an investigation of the effects of treatments in cystic fibrosis using the UK Cystic Fibrosis Registry data focussing on two outcomes: lung function (continuous outcome) and annual number of days receiving intravenous antibiotics (count outcome). We identified five features of this data that may affect the performance of the methods: misspecification of the causal null, long-term treatment effects, effect modification by time-varying covariates, misspecification of the direction of causal pathways, and censoring. In simulation studies, under ideal settings, all five methods provide consistent estimates of the treatment effect with little difference between methods. However, all methods performed poorly under some settings, highlighting the importance of using appropriate methods based on the data available. Furthermore, with the count outcome, the issue of non-collapsibility makes comparison between methods delivering marginal and conditional effects difficult. In many situations, we would recommend using more than one of the available methods for analysis, as if the effect estimates are very different, this would indicate potential issues with the analyses.

SUBMITTER: Newsome SJ

PROVIDER: S-EPMC6001810 | biostudies-other | 2018 Jul

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty.

Newsome Simon J SJ Keogh Ruth H RH Daniel Rhian M RM

Statistics in medicine 20180419 15

In the presence of time-dependent confounding, there are several methods available to estimate treatment effects. With correctly specified models and appropriate structural assumptions, any of these methods could provide consistent effect estimates, but with real-world data, all models will be misspecified and it is difficult to know if assumptions are violated. In this paper, we investigate five methods: inverse probability weighting of marginal structural models, history-adjusted marginal stru ...[more]

PMID: 29671915

Similar Datasets

Project description:BackgroundManaging diagnostic uncertainty is a major challenge in primary care due to factors such as the absence of definitive tests, variable symptom presentations and disease evolution. Maintaining patient trust during a period of investigative uncertainty, whilst minimising scope for diagnostic error is a challenge. Mismanagement can lead to diagnostic errors, treatment delays, and suboptimal patient outcomes.ObjectiveOur aim was to explore how UK primary care physicians (GPs) address and communicate diagnostic uncertainty in practice.DesignThis qualitative study used video and audio-recordings. Verbatim transcripts were coded with a modified, validated tool to capture GPs' actions and communication in primary care consultations that included diagnostic uncertainty. The tool includes items relating to advice regarding new symptoms or symptom deterioration (sometimes called 'safety netting'). Video data was analysed to identify GP and patient body postures during and after the delivery of the management plan.ParticipantsAll patient participants had a consultation with a GP, were over the age of 50 and had (1) at least one new presenting problem or (2) one persistent problem that was undiagnosed.ApproachData collection occurred in GP-patient consultations during 2017-2018 across 7 practices in UK during 2017-2018.Key resultsGPs used various management strategies to address diagnostic uncertainty, including (1) symptom monitoring without treatment, (2) prescribed treatment with symptom monitoring, and (3) addressing risks that could arise from administrative tasks. GPs did not make management plans for potential treatment side effects. Specificity of uncertainty management plans varied among GPs, with only some offering detailed actions and timescales. The transfer of responsibility for the management plan to patients was usually delivered rather than negotiated, with most patients confirming acceptance before concluding the discussion.ConclusionsWe offer guidance to healthcare professionals, improving awareness of using and communicating management plans for diagnostic uncertainty.

Project description:BackgroundTreatment switching in randomised controlled trials (RCTs) is a problem for health technology assessment when substantial proportions of patients switch onto effective treatments that would not be available in standard clinical practice. Often statistical methods are used to adjust for switching: these can be applied in different ways, and performance has been assessed in simulation studies, but not in real-world case studies. We assessed the performance of adjustment methods described in National Institute for Health and Care Excellence Decision Support Unit Technical Support Document 16, applying them to an RCT comparing panitumumab to best supportive care (BSC) in colorectal cancer, in which 76% of patients randomised to BSC switched onto panitumumab. The RCT resulted in intention-to-treat hazard ratios (HR) for overall survival (OS) of 1.00 (95% confidence interval [CI] 0.82-1.22) for all patients, and 0.99 (95% CI 0.75-1.29) for patients with wild-type KRAS (Kirsten rat sarcoma virus).MethodsWe tested several applications of inverse probability of censoring weights (IPCW), rank preserving structural failure time models (RPSFTM) and simple and complex two-stage estimation (TSE) to estimate treatment effects that would have been observed if BSC patients had not switched onto panitumumab. To assess the performance of these analyses we ascertained the true effectiveness of panitumumab based on: (i) subsequent RCTs of panitumumab that disallowed treatment switching; (ii) studies of cetuximab that disallowed treatment switching, (iii) analyses demonstrating that only patients with wild-type KRAS benefit from panitumumab. These sources suggest the true OS HR for panitumumab is 0.76-0.77 (95% CI 0.60-0.98) for all patients, and 0.55-0.73 (95% CI 0.41-0.93) for patients with wild-type KRAS.ResultsSome applications of IPCW and TSE provided treatment effect estimates that closely matched the point-estimates and CIs of the expected truths. However, other applications produced estimates towards the boundaries of the expected truths, with some TSE applications producing estimates that lay outside the expected true confidence intervals. The RPSFTM performed relatively poorly, with all applications providing treatment effect estimates close to 1, often with extremely wide confidence intervals.ConclusionsAdjustment analyses may provide unreliable results. How each method is applied must be scrutinised to assess reliability.

Dataset Information

Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty.

Publications

Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets