Dataset Information

Why do authors derive new cardiovascular clinical prediction rules in the presence of existing rules? A mixed methods study.

ABSTRACT:

Background

Researchers should examine existing evidence to determine the need for a new study. It is unknown whether developers evaluate existing evidence to justify new cardiovascular clinical prediction rules (CPRs).

Objective

We aimed to assess whether authors of cardiovascular CPRs cited existing CPRs, why some authors did not cite existing CPRs, and why they thought existing CPRs were insufficient.

Method

Derivation studies of cardiovascular CPRs from the International Register of Clinical Prediction Rules for Primary Care were evaluated. We reviewed the introduction sections to determine whether existing CPRs were cited. Using thematic content analysis, the stated reasons for determining existing cardiovascular CPRs insufficient were explored. Study authors were surveyed via e-mail and post. We asked whether they were aware of any existing cardiovascular CPRs at the time of derivation, how they searched for existing CPRs, and whether they thought it was important to cite existing CPRs.

Results

Of 85 derivation studies included, 48 (56.5%) cited existing CPRs, 33 (38.8%) did not cite any CPR, and four (4.7%) declared there was none to cite. Content analysis identified five categories of existing CPRs insufficiency related to: (1) derivation (5 studies; 11.4% of 44), (2) construct (31 studies; 70.5%), (3) performance (10 studies; 22.7%), (4) transferability (13 studies; 29.5%), and (5) evidence (8 studies; 18.2%). Authors of 54 derivation studies (71.1% of 76 authors contacted) responded to the survey. Twenty-five authors (46.3%) reported they were aware of existing CPR at the time of derivation. Twenty-nine authors (53.7%) declared they conducted a systematic search to identify existing CPRs. Most authors (90.7%) indicated citing existing CPRs was important.

Conclusion

Cardiovascular CPRs are often developed without citing existing CPRs although most authors agree it is important. Common justifications for new CPRs concerned construct, including choice of predictor variables or relevance of outcomes. Developers should clearly justify why new CPRs are needed with reference to existing CPRs to avoid unnecessary duplication.

SUBMITTER: Ban JW

PROVIDER: S-EPMC5462434 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Why do authors derive new cardiovascular clinical prediction rules in the presence of existing rules? A mixed methods study.

Ban Jong-Wook JW Wallace Emma E Stevens Richard R Perera Rafael R

PloS one 20170607 6

<h4>Background</h4>Researchers should examine existing evidence to determine the need for a new study. It is unknown whether developers evaluate existing evidence to justify new cardiovascular clinical prediction rules (CPRs).<h4>Objective</h4>We aimed to assess whether authors of cardiovascular CPRs cited existing CPRs, why some authors did not cite existing CPRs, and why they thought existing CPRs were insufficient.<h4>Method</h4>Derivation studies of cardiovascular CPRs from the International ...[more]

PMID: 28591223

Similar Datasets

Project description:ObjectivesTo develop prediction models that better estimate the pretest probability of coronary artery disease in low prevalence populations.DesignRetrospective pooled analysis of individual patient data.Setting18 hospitals in Europe and the United States.ParticipantsPatients with stable chest pain without evidence for previous coronary artery disease, if they were referred for computed tomography (CT) based coronary angiography or catheter based coronary angiography (indicated as low and high prevalence settings, respectively).Main outcome measuresObstructive coronary artery disease (≥ 50% diameter stenosis in at least one vessel found on catheter based coronary angiography). Multiple imputation accounted for missing predictors and outcomes, exploiting strong correlation between the two angiography procedures. Predictive models included a basic model (age, sex, symptoms, and setting), clinical model (basic model factors and diabetes, hypertension, dyslipidaemia, and smoking), and extended model (clinical model factors and use of the CT based coronary calcium score). We assessed discrimination (c statistic), calibration, and continuous net reclassification improvement by cross validation for the four largest low prevalence datasets separately and the smaller remaining low prevalence datasets combined.ResultsWe included 5677 patients (3283 men, 2394 women), of whom 1634 had obstructive coronary artery disease found on catheter based coronary angiography. All potential predictors were significantly associated with the presence of disease in univariable and multivariable analyses. The clinical model improved the prediction, compared with the basic model (cross validated c statistic improvement from 0.77 to 0.79, net reclassification improvement 35%); the coronary calcium score in the extended model was a major predictor (0.79 to 0.88, 102%). Calibration for low prevalence datasets was satisfactory.ConclusionsUpdated prediction models including age, sex, symptoms, and cardiovascular risk factors allow for accurate estimation of the pretest probability of coronary artery disease in low prevalence populations. Addition of coronary calcium scores to the prediction models improves the estimates.

Project description:Alcohol consumption is a major cause of mortality and morbidity globally. In response to strong calls from the public for alcohol law reform, the New Zealand Government recently reduced the blood alcohol limit for driving and introduced the Sale and Supply of Alcohol Act which aim to (1) improve community input into local decision-making on alcohol; (2) reduce the availability of alcohol; and (3) reduce hazardous drinking and alcohol-related harm. In this project we seek to evaluate the new laws in terms of these objectives.A policy evaluation framework is proposed to investigate the implementation and outcomes of the reforms. We will use quantitative and qualitative methods, employing a pre-post design. Participants include members of the public, local government staff, iwi (Māori tribal groups that function collectively to support their members) and community group representatives. Data will be collected via postal surveys, interviews and analysis of local government documents. Liquor licensing, police and hospital injury data will also be used. Community input into local government decision-making will be operationalised as: the number of objections per license application and the number of local governments adopting a local alcohol policy (LAP). Outcome measures will be the 'restrictiveness' of LAPs compared to previous policies, the number (per 1000 residents) and density (per square kilometre) of alcohol outlets throughout NZ, and the number of weekend late-night (i.e., post 10 pm) trading hours. For consumption and harm, outcomes will be the prevalence of hazardous drinking, harm from own and others' drinking, community amenity effects, rates of assault, and rates of alcohol-involved traffic crashes. Multiple regression will be used to model how the outcomes vary by local government area from before to after the law changes take effect. These measures will be complemented by qualitative analysis of LAP development and public participation in local decision-making on alcohol.The project will evaluate how well the reforms meet their explicit public health objectives.

Project description:BACKGROUND:Recent discoveries in the fields of machine learning (ML), Ecological Momentary Assessment (EMA), computerized adaptive testing (CAT), digital phenotype, imaging, and biomarkers have brought about a new paradigm shift in medicine. OBJECTIVE:The aim of this study was to explore psychiatrists' perspectives on this paradigm through the prism of new clinical decision support systems (CDSSs). Our primary objective was to assess the acceptability of these new technologies. Our secondary objective was to characterize the factors affecting their acceptability. METHODS:A sample of psychiatrists was recruited through a mailing list. Respondents completed a Web-based survey. A quantitative study with an original form of assessment involving the screenplay method was implemented involving 3 scenarios, each featuring 1 of the 3 support systems, namely, EMA and CAT, biosensors comprising a connected wristband-based digital phenotype, and an ML-based blood test or magnetic resonance imaging (MRI). We investigated 4 acceptability domains based on International Organization for Standardization and Nielsen models (usefulness, usability, reliability, and risk). RESULTS:We recorded 515 observations. Regarding our primary objective, overall acceptability was moderate. MRI coupled with ML was considered to be the most useful system, and the connected wristband was considered the least. All the systems were described as risky (410/515, 79.6%). Regarding our secondary objective, acceptability was strongly influenced by socioepidemiological variables (professional culture), such as gender, age, and theoretical approach. CONCLUSIONS:This is the first study to assess psychiatrists' views on new CDSSs. Data revealed moderate acceptability, but our analysis shows that this is more the result of the lack of knowledge about these new technologies rather than a strong rejection. Furthermore, we found strong correspondences between acceptability profiles and professional culture profiles. Many medical, forensics, and ethical issues were raised, including therapeutic relationship, data security, data storage, and privacy risk. It is essential for psychiatrists to receive training and become involved in the development of new technologies.

Project description:BackgroundClinical prediction rules (CPRs) should be externally validated by independent researchers. Although there are many cardiovascular CPRs, most have not been externally validated. It is not known why some CPRs are externally validated by independent researchers and others are not.MethodsWe analyzed cardiovascular risk CPRs included in a systematic review. Independent external validations were identified by forward citation searches of derivation studies. Time between the publication of a cardiovascular CPR and the first independent external validation was calculated. We assessed Kaplan-Meier estimates of the probability to have an independent external validation. Using univariable Cox regression, we explored whether characteristics of derivation (design, location, sample size, number of predictors, presentation format, validation in derivation), reporting (participants, predictors, outcomes, performance measure, information for risk calculation), and publication (journal impact factor) are associated with time to the first independent external validation.ResultsOf 125 cardiovascular risk CPRs, 29 had an independent external validation. The median follow-up was 118 months (95% CI, 99-130). The 25th percentile of event time was 122 months (95% CI, 91-299). Cardiovascular risk CPRs from the USA were 4.15 times (95% CI, 1.89-9.13) more likely to have an independent external validation. Increasing the sample size of derivation by ten times was associated with a 2.32-fold (95% CI, 1.37-3.91) increase in the probability of having an independent external validation. CPRs presented with an internal validation tend to get an independent external validation sooner (HR = 1.73, 95% CI, 0.77-3.93). CPRs reporting all the information necessary for calculating individual risk were 2.65 (95% CI, 1.01-6.96) times more likely to have an independent external validation. Publishing a cardiovascular risk CPR in a journal that has one unit higher impact factor was associated with a 6% (95% CI, 3-9) higher likelihood of an independent external validation.ConclusionsThe probability for cardiovascular risk CPRs to get an independent external validation was low even many years after their derivations. Authors of new cardiovascular risk CPRs should consider using adequate sample size, conducting an internal validation, and reporting all the information needed for risk calculation as these features were associated with an independent external validation.

Dataset Information

Why do authors derive new cardiovascular clinical prediction rules in the presence of existing rules? A mixed methods study.

Background

Objective

Method

Results

Conclusion

Publications

Why do authors derive new cardiovascular clinical prediction rules in the presence of existing rules? A mixed methods study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure