Dataset Information

Estimating individual treatment effects on COPD exacerbations by causal machine learning on randomised controlled trials.

ABSTRACT:

Rationale

Estimating the causal effect of an intervention at individual level, also called individual treatment effect (ITE), may help in identifying response prior to the intervention.

Objectives

We aimed to develop machine learning (ML) models which estimate ITE of an intervention using data from randomised controlled trials and illustrate this approach with prediction of ITE on annual chronic obstructive pulmonary disease (COPD) exacerbation rates.

Methods

We used data from 8151 patients with COPD of the Study to Understand Mortality and MorbidITy in COPD (SUMMIT) trial (NCT01313676) to address the ITE of fluticasone furoate/vilanterol (FF/VI) versus control (placebo) on exacerbation rate and developed a novel metric, Q-score, for assessing the power of causal inference models. We then validated the methodology on 5990 subjects from the InforMing the PAthway of COPD Treatment (IMPACT) trial (NCT02164513) to estimate the ITE of FF/umeclidinium/VI (FF/UMEC/VI) versus UMEC/VI on exacerbation rate. We used Causal Forest as causal inference model.

Results

In SUMMIT, Causal Forest was optimised on the training set (n=5705) and tested on 2446 subjects (Q-score 0.61). In IMPACT, Causal Forest was optimised on 4193 subjects in the training set and tested on 1797 individuals (Q-score 0.21). In both trials, the quantiles of patients with the strongest ITE consistently demonstrated the largest reductions in observed exacerbations rates (0.54 and 0.53, p<0.001). Poor lung function and blood eosinophils, respectively, were the strongest predictors of ITE.

Conclusions

This study shows that ML models for causal inference can be used to identify individual response to different COPD treatments and highlight treatment traits. Such models could become clinically useful tools for individual treatment decisions in COPD.

SUBMITTER: Verstraete K

PROVIDER: S-EPMC10511983 | biostudies-literature | 2023 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Estimating individual treatment effects on COPD exacerbations by causal machine learning on randomised controlled trials.

Verstraete Kenneth K Gyselinck Iwein I Huts Helene H Das Nilakash N Topalovic Marko M De Vos Maarten M Janssens Wim W

Thorax 20230403 10

<h4>Rationale</h4>Estimating the causal effect of an intervention at individual level, also called individual treatment effect (ITE), may help in identifying response prior to the intervention.<h4>Objectives</h4>We aimed to develop machine learning (ML) models which estimate ITE of an intervention using data from randomised controlled trials and illustrate this approach with prediction of ITE on annual chronic obstructive pulmonary disease (COPD) exacerbation rates.<h4>Methods</h4>We used data f ...[more]

PMID: 37012070

Similar Datasets

Project description:ObjectivesTo predict treatment effects for individual patients based on data from randomised trials, taking rosuvastatin treatment in the primary prevention of cardiovascular disease as an example, and to evaluate the net benefit of making treatment decisions for individual patients based on a predicted absolute treatment effect.SettingAs an example, data were used from the Justification for the Use of Statins in Prevention (JUPITER) trial, a randomised controlled trial evaluating the effect of rosuvastatin 20 mg daily versus placebo on the occurrence of cardiovascular events (myocardial infarction, stroke, arterial revascularisation, admission to hospital for unstable angina, or death from cardiovascular causes). Population 17,802 healthy men and women who had low density lipoprotein cholesterol levels of less than 3.4 mmol/L and high sensitivity C reactive protein levels of 2.0 mg/L or more.MethodsData from the Justification for the Use of Statins in Prevention trial were used to predict rosuvastatin treatment effect for individual patients based on existing risk scores (Framingham and Reynolds) and on a newly developed prediction model. We compared the net benefit of prediction based rosuvastatin treatment (selective treatment of patients whose predicted treatment effect exceeds a decision threshold) with the net benefit of treating either everyone or no one.ResultsThe median predicted 10 year absolute risk reduction for cardiovascular events was 4.4% (interquartile range 2.6-7.0%) based on the Framingham risk score, 4.2% (2.5-7.1%) based on the Reynolds score, and 3.9% (2.5-6.1%) based on the newly developed model (optimal fit model). Prediction based treatment was associated with more net benefit than treating everyone or no one, provided that the decision threshold was between 2% and 7%, and thus that the number willing to treat (NWT) to prevent one cardiovascular event over 10 years was between 15 and 50.ConclusionsData from randomised trials can be used to predict treatment effect in terms of absolute risk reduction for individual patients, based on a newly developed model or, if available, existing risk scores. The value of such prediction of treatment effect for medical decision making is conditional on the NWT to prevent one outcome event. Trial registration number Clinicaltrials.gov NCT00239681.

Project description:IntroductionAlthough clinical, functional, and biomarker data predict asthma exacerbations, newer approaches providing high accuracy of prognosis are needed for real-world decision-making in asthma. Machine learning (ML) leverages mathematical and statistical methods to detect patterns for future disease events across large datasets from electronic health records (EHR). This study conducted training and fine-tuning of ML algorithms for the real-world prediction of asthma exacerbations in patients with physician-diagnosed asthma.MethodsAdults with ≥ 2 ICD9/10 asthma codes within 1 year and at least 30 days apart were identified from the Optum Panther EHR database between 2016 and 2023. An emergency department (ED), urgent care, or inpatient visit for asthma, while on systemic administration of corticosteroids, was considered an exacerbation. To predict factors associated with exacerbations in a 6-month study period, clinical information from patients was retrieved in the preceding 6-month baseline period. Clinical information included demographics, lab results, diagnoses, medications, immunizations, and allergies. Three models built using Extreme Gradient Boosting (XGBoost), Long Short-Term Memory (LSTM), and Transformers algorithms were trained and tested on independent datasets. Predictions were explained using the SHAP (SHapley Additive exPlanations) library.ResultsOf 1,331,934 patients with asthma, 16,279 (1.2%) experienced ≥ 1 exacerbation. XGBoost was the best predictive algorithm (area under the curve [AUC] = 0.964). Factors associated with exacerbations included a prior history of exacerbation, prednisone usage, high-dose albuterol usage, and elevated troponin I. Reduced probability of exacerbations was associated with receiving inhaled albuterol, vitamins, aspirin, statins, furosemide, and influenza vaccination.ConclusionThis ML-based study on asthma in the real world confirmed previously known features associated with increased exacerbation risk for asthma, while uncovering not entirely understood features associated with reduced risk of asthma exacerbations. These findings are hypothesis-generating and should contribute to ongoing discussion of the strengths and limitations of ML and other supervised learning models in patient risk stratification.

Project description:ObjectiveRandomised controlled trials (RCTs) are often considered as the gold standard for assessing new health interventions. Patients are randomly assigned to receive an intervention or control. The effect of the intervention can be estimated by comparing outcomes between groups, whose prognostic factors are expected to balance by randomisation. However, patients' non-compliance with their assigned treatment will undermine randomisation and potentially bias the estimate of treatment effect. Through simulation, we aim to compare common approaches in analysing non-compliant data under different non-compliant scenarios.SettingsBased on a real study, we simulated hypothetical trials by varying three non-compliant factors: the type, randomness and degree of non-compliance. We compared the intention-to-treat (ITT), as-treated (AT), per-protocol (PP), instrumental variable (IV) and complier average casual effect (CACE) analyses to estimate large (50% improvement over the control), moderate (25% improvement) and null (same as the control) treatment effects. Different approaches were compared by the bias of estimate, mean square error (MSE) and 95% coverage of the true value.ResultsFor a large or moderate treatment effect, the ITT estimate was considerably biased in all scenarios. The AT, PP, IV and CACE estimates were unbiased when non-compliant behaviours were random. The IV estimate was unbiased when non-compliant behaviours were symmetrically dependent on patients' conditions. The PP estimate was mostly unbiased when patients in the control group did not have access to the intervention. When the intervention was not different from the control, the ITT was less biased than the other approaches. Similar results were found when comparing the MSE and 95% coverage.ConclusionsThe standard ITT analysis under non-compliance is biased when the intervention has a moderate or large effect. Alternative analyses can provide unbiased or less biased estimates. Based on the results, we make some suggestions on choosing optimal approaches for analysing specific non-compliant scenarios.

Project description:BackgroundRetrospective exploratory analyses of randomised controlled trials (RCTs) seeking to identify treatment effect heterogeneity (TEH) are prone to bias and false positives. Yet the desire to learn all we can from exhaustive data measurements on trial participants motivates the inclusion of such analyses within RCTs. Moreover, widespread advances in machine learning (ML) methods hold potential to utilise such data to identify subjects exhibiting heterogeneous treatment response.MethodsWe present a novel analysis strategy for detecting TEH in randomised data using ML methods, whilst ensuring proper control of the false positive discovery rate. Our approach uses random data partitioning with statistical or ML-based prediction on held-out data. This method can test for both crossover TEH (switch in optimal treatment) and non-crossover TEH (systematic variation in benefit across patients). The former is done via a two-sample hypothesis test measuring overall predictive performance. The latter is done via 'stacking' the ML predictors alongside a classical statistical model to formally test the added benefit of the ML algorithm. An adaptation of recent statistical theory allows for the construction of a valid aggregate p value. This testing strategy is independent of the choice of ML method.ResultsWe demonstrate our approach with a re-analysis of the SEAQUAMAT trial, which compared quinine to artesunate for the treatment of severe malaria in Asian adults. We find no evidence for any subgroup who would benefit from a change in treatment from the current standard of care, artesunate, but strong evidence for significant TEH within the artesunate treatment group. In particular, we find that artesunate provides a differential benefit to patients with high numbers of circulating ring stage parasites.ConclusionsML analysis plans using computational notebooks (documents linked to a programming language that capture the model parameter settings, data processing choices, and evaluation criteria) along with version control can improve the robustness and transparency of RCT exploratory analyses. A data-partitioning algorithm allows researchers to apply the latest ML techniques safe in the knowledge that any declared associations are statistically significant at a user-defined level.

Dataset Information

Estimating individual treatment effects on COPD exacerbations by causal machine learning on randomised controlled trials.

Rationale

Objectives

Methods

Results

Conclusions

Publications

Estimating individual treatment effects on COPD exacerbations by causal machine learning on randomised controlled trials.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets