Dataset Information

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council.

ABSTRACT: Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of crucial responsibilities that need to be completed when developing ML models. This paper aims to serve as a scientific foundation to aid investigators, data scientists, authors, editors, and reviewers involved in machine learning research with the intent of uniform reporting of ML investigations. An independent multidisciplinary panel of ML experts, clinicians, and statisticians worked together to review the theoretical rationale underlying 7 sets of requirements that may reduce algorithmic errors and biases. Finally, the paper summarizes a list of reporting items as an itemized checklist that highlights steps for ensuring correct application of ML models and the consistent reporting of model specifications and results. It is expected that the rapid pace of research and development and the increased availability of real-world evidence may require periodic updates to the checklist.

SUBMITTER: Sengupta PP

PROVIDER: S-EPMC7953597 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council.

Sengupta Partho P PP Shrestha Sirish S Berthon Béatrice B Messas Emmanuel E Donal Erwan E Tison Geoffrey H GH Min James K JK D'hooge Jan J Voigt Jens-Uwe JU Dudley Joel J Verjans Johan W JW Shameer Khader K Johnson Kipp K Lovstakken Lasse L Tabassian Mahdi M Piccirilli Marco M Pernot Mathieu M Yanamala Naveena N Duchateau Nicolas N Kagiyama Nobuyuki N Bernard Olivier O Slomka Piotr P Deo Rahul R Arnaout Rima R

JACC. Cardiovascular imaging 20200901 9

Machine learning (ML) has been increasingly used within cardiology, particularly in the domain of cardiovascular imaging. Due to the inherent complexity and flexibility of ML algorithms, inconsistencies in the model performance and interpretation may occur. Several review articles have been recently published that introduce the fundamental principles and clinical application of ML for cardiologists. This paper builds on these introductory principles and outlines a more comprehensive list of cruc ...[more]

PMID: 32912474

Similar Datasets

Project description:ImportanceThe 2013 American College of Cardiology/American Heart Association (ACC/AHA) Cholesterol Management Guideline recommends moderate-intensity to high-intensity statin therapy in eligible patients.ObjectiveTo examine adoption of the 2013 ACC/AHA guideline in US cardiology practices.Design, setting, and participantsAmong 161 cardiology practices, trends in the use of moderate-intensity to high-intensity statin and nonstatin lipid-lowering therapy (LLT) were analyzed before (September 1, 2012, to November 1, 2013) and after (February 1, 2014, to April 1, 2015) publication of the 2013 ACC/AHA guideline among 4 mutually exclusive risk groups within the ACC Practice Innovation and Clinical Excellence Registry. Interrupted time series analysis was used to evaluate for differences in trend in use of moderate-intensity to high-intensity statin and nonstatin LLT use in hierarchical logistic regression models. Participants were a population-based sample of 1 105 356 preguideline patients (2 431 192 patient encounters) and 1 116 472 postguideline patients (2 377 219 patient encounters). Approximately 97% of patients had atherosclerotic cardiovascular disease (ASCVD).ExposuresModerate-intensity to high-intensity statin and nonstatin LLT use before and after publication of the 2013 ACC/AHA guideline.Main outcomes and measuresTime trend in the use of moderate-intensity to high-intensity statin and nonstatin LLT.ResultsIn the study cohort, the mean (SD) age was 69.6 (12.1) years among 1 105 356 patients (40.2% female) before publication of the guideline and 70.0 (11.9) years among 1 116 472 patients (39.8% female) after publication of the guideline. Although there was a trend toward increasing use of moderate-intensity to high-intensity statins overall and in the ASCVD cohort, such a trend was already present before publication of the guideline. No significant difference in trend in the use of moderate-intensity to high-intensity statins was observed in other groups. The use of moderate-intensity to high-intensity statin therapy was 62.1% (before publication of the guideline) and 66.6% (after publication of the guideline) in the overall cohort, 62.7% (before publication) and 67.0% (after publication) in the ASCVD cohort, 50.6% (before publication) and 52.3% (after publication) in the cohort with elevated low-density lipoprotein cholesterol levels (ie, ≥190 mg/dL), 52.4% (before publication) and 55.2% (after publication) in the diabetes cohort, and 41.9% (before publication) and 46.9% (after publication) in the remaining group with 10-year ASCVD risk of 7.5% or higher. In hierarchical logistic regression models, there was a significant increase in the use of moderate-intensity to high-intensity statins in the overall cohort (4.8%) and in the ASCVD cohort (4.3%) (P < .01 for slope for both). There was no significant change for other risk cohorts. Nonstatin LLT use remained unchanged in the preguideline and postguideline periods in the hierarchical logistic regression models for all of the risk groups.Conclusions and relevanceAdoption of the 2013 ACC/AHA Cholesterol Management Guideline in cardiology practices was modest. Timely interventions are needed to improve guideline-concordant practice to reduce the burden of ASCVD.

Project description:ImportanceClinical decisions are ideally based on evidence generated from multiple randomized controlled trials (RCTs) evaluating clinical outcomes, but historically, few clinical guideline recommendations have been based entirely on this type of evidence.ObjectiveTo determine the class and level of evidence (LOE) supporting current major cardiovascular society guideline recommendations, and changes in LOE over time.Data sourcesCurrent American College of Cardiology/American Heart Association (ACC/AHA) and European Society of Cardiology (ESC) clinical guideline documents (2008-2018), as identified on cardiovascular society websites, and immediate predecessors to these guideline documents (1999-2014), as referenced in current guideline documents.Study selectionComprehensive guideline documents including recommendations organized by class and LOE.Data extraction and synthesisThe number of recommendations and the distribution of LOE (A [supported by data from multiple RCTs or a single, large RCT], B [supported by data from observational studies or a single RCT], and C [supported by expert opinion only]) were determined for each guideline document.Main outcomes and measuresThe proportion of guideline recommendations supported by evidence from multiple RCTs (LOE A).ResultsAcross 26 current ACC/AHA guidelines (2930 recommendations; median, 121 recommendations per guideline [25th-75th percentiles, 76-155]), 248 recommendations (8.5%) were classified as LOE A, 1465 (50.0%) as LOE B, and 1217 (41.5%) as LOE C. The median proportion of LOE A recommendations was 7.9% (25th-75th percentiles, 0.9%-15.2%). Across 25 current ESC guideline documents (3399 recommendations; median, 130 recommendations per guideline [25th-75th percentiles, 111-154]), 484 recommendations (14.2%) were classified as LOE A, 1053 (31.0%) as LOE B, and 1862 (54.8%) as LOE C. When comparing current guidelines with prior versions, the proportion of recommendations that were LOE A did not increase in either ACC/AHA (median, 9.0% [current] vs 11.7% [prior]) or ESC guidelines (median, 15.1% [current] vs 17.6% [prior]).Conclusions and relevanceAmong recommendations in major cardiovascular society guidelines, only a small percentage were supported by evidence from multiple RCTs or a single, large RCT. This pattern does not appear to have meaningfully improved from 2008 to 2018.

Project description:We examined compliance with performance measures for 14,464 patients enrolled from July 2008 through June 2009 into the American College of Cardiology's PINNACLE (Practice Innovation And Clinical Excellence) program to provide initial insights into the quality of outpatient cardiac care.Little is known about the quality of care of outpatients with coronary artery disease (CAD), heart failure, and atrial fibrillation, and whether sex and racial disparities exist in the treatment of outpatients.The PINNACLE program is the first, national, prospective office-based quality improvement program of cardiac patients designed, in part, to capture, report, and improve outpatient performance measure compliance. We examined the proportion of patients whose care was compliant with established American College of Cardiology, American Heart Association, and American Medical Association-Physician Consortium for Performance Improvement (ACC/AHA/PCPI) performance measures for CAD, heart failure, and atrial fibrillation.There were 14,464 unique patients enrolled from 27 U.S. practices, accounting for 18,021 clinical visits. Of these, 8,132 (56.4%) had CAD, 5,012 (34.7%) had heart failure, and 2,786 (19.3%) had nonvalvular atrial fibrillation. Data from the PINNACLE program were feasibly collected for 24 of 25 ACC/AHA/PCPI performance measures. Compliance with performance measures ranged from being very low (e.g., 13.3% of CAD patients screened for diabetes mellitus) to very high (e.g., 96.7% of heart failure patients with blood pressure assessments), with moderate (70% to 90%) compliance observed for most performance measures. For 3 performance measures, there were small differences in compliance rates by race or sex.For more than 14,000 patients enrolled from 27 practices in the outpatient PINNACLE program, we found that compliance with performance measures was variable, even after accounting for exclusion criteria, suggesting an important opportunity to improve the quality of outpatient care.

Project description:ImportanceLittle is known regarding the durability of clinical practice guideline recommendations over time.ObjectiveTo characterize variations in the durability of class I ("procedure/treatment should be performed/administered") American College of Cardiology/American Heart Association (ACC/AHA) guideline recommendations.Design, setting, and participantsTextual analysis by 4 independent reviewers of 11 guidelines published between 1998 and 2007 and revised between 2006 and 2013.Main outcomes and measuresWe abstracted all class I recommendations from the first of the 2 most recent versions of each guideline and identified corresponding recommendations in the subsequent version. We classified recommendations replaced by less determinate or contrary recommendations as having been downgraded or reversed; we classified recommendations for which no corresponding item could be identified as having been omitted. We tested for differences in the durability of recommendations according to guideline topic and underlying level of evidence using bivariable hypothesis tests and conditional logistic regression.ResultsOf 619 index recommendations, 495 (80.0%; 95% CI, 76.6%-83.1%) were retained in the subsequent guideline version, 57 (9.2%; 95% CI, 7.0%-11.8%) were downgraded or reversed, and 67 (10.8%; 95% CI, 8.4%-13.3%) were omitted. The percentage of recommendations retained varied across guidelines from 15.4% (95% CI, 1.9%-45.4%) to 94.1% (95% CI, 80.3%-99.3%; P < .001). Among recommendations with available information on level of evidence, 90.5% (95% CI, 83.2%-95.3%) of recommendations supported by multiple randomized studies were retained, vs 81.0% (95% CI, 74.8%-86.3%) of recommendations supported by 1 randomized trial or observational data and 73.7% (95% CI, 65.8%-80.5%) of recommendations supported by opinion (P = .001). After accounting for guideline-level factors, the probability of being downgraded, reversed, or omitted was greater for recommendations based on opinion (odds ratio, 3.14; 95% CI, 1.69-5.85; P < .001) or on 1 trial or observational data (odds ratio, 3.49; 95% CI, 1.45-8.41; P = .005) vs recommendations based on multiple trials.Conclusions and relevanceThe durability of class I cardiology guideline recommendations for procedures and treatments promulgated by the ACC/AHA varied across individual guidelines and levels of evidence. Downgrades, reversals, and omissions were most common among recommendations not supported by multiple randomized studies.

Project description:ObjectivesThe AHA/ACC-2017 hypertension guideline recommends an age-independent target blood pressure (BP) of less than 130/80 mmHg. In an elderly cohort without established cardiovascular disease (CVD) at baseline, we determined the impact of this guideline on the prevalence of hypertension and associated CVD risk.MethodsNineteen thousand, one hundred and fourteen participants aged at least 65 years from the ASPirin in Reducing Events in the Elderly (ASPREE) study were grouped by baseline BP: 'pre-2017 hypertensive' (BP ≥140/90 mmHg and/or on antihypertensive drugs); 'reclassified hypertensive' (normotensive by pre-2017 guidelines; hypertensive by AHA/ACC-2017 guideline), and 'normotensive' (BP <130 and <80 mmHg). For each group, we evaluated CVD risk factors, predicted 10-year CVD risk using the Atherosclerotic Cardiovascular Disease (ASCVD) risk equation, and reported observed CVD event rates during a median 4.7-year follow-up.ResultsOverall, 74.4% (14 213/19 114) were 'pre-2017 hypertensive'; an additional 12.3% (2354/19 114) were 'reclassified hypertensive' by the AHA/ACC-2017 guideline. Of those 'reclassified hypertensive', the majority (94.5%) met criteria for antihypertensive treatment although 29% had no other traditional CVD risk factors other than age. Further, a relatively lower mean 10-year predicted CVD risk (18% versus 26%, P < 0.001) and lower CVD rates (8.9 versus 12.1/1000 person-years, P = 0.01) were observed in 'reclassified hypertensive' compared with 'pre-2017 hypertensive'. Compared with 'normotensive', a hazard ratio (95% confidence interval) for CVD events of 1.60 (1.26-2.02) for 'pre-2017 hypertensive' and 1.26 (0.93-1.71) for 'reclassified hypertensive' was observed.ConclusionApplying current CVD risk calculators in the elderly 'reclassified hypertensive', as a result of shifting the BP threshold lower, increases eligibility for antihypertensive treatment but documented CVD rates remain lower than hypertensive patients defined by pre2017 BP thresholds.

Dataset Information

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council.

Publications

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets