The perils of meta-regression to identify clinical decision support system success factors.
Ontology highlight
ABSTRACT:
SUBMITTER: Fillmore CL
PROVIDER: S-EPMC4866642 | biostudies-literature | 2015 Aug
REPOSITORIES: biostudies-literature
Publications
The perils of meta-regression to identify clinical decision support system success factors.
Fillmore Christopher L CL Rommel Casey A CA Welch Brandon M BM Zhang Mingyuan M Kawamoto Kensaku K
Journal of biomedical informatics 20150518
Clinical decision support interventions are typically heterogeneous in nature, making it difficult to identify why some interventions succeed while others do not. One approach to identify factors important to the success of health information systems is the use of meta-regression techniques, in which potential explanatory factors are correlated with the outcome of interest. This approach, however, can result in misleading conclusions due to several issues. In this manuscript, we present a cautio ...[more]
Similar Datasets
Project description:BackgroundComputerized clinical decision support systems (CDSSs) are a promising knowledge translation tool, but often fail to meaningfully influence the outcomes they target. Low CDSS provider uptake is a potential contributor to this problem but has not been systematically studied. The objective of this systematic review and meta-regression was to determine reported CDSS uptake and identify which CDSS features may influence uptake.MethodsMedline, Embase, CINAHL, and the Cochrane Database of Controlled Trials were searched from January 2000 to August 2020. Randomized, non-randomized, and quasi-experimental trials reporting CDSS uptake in any patient population or setting were included. The main outcome extracted was CDSS uptake, reported as a raw proportion, and representing the number of times the CDSS was used or accessed over the total number of times it could have been interacted with. We also extracted context, content, system, and implementation features that might influence uptake, for each CDSS. Overall weighted uptake was calculated using random-effects meta-analysis and determinants of uptake were investigated using multivariable meta-regression.ResultsAmong 7995 citations screened, 55 studies involving 373,608 patients and 3607 providers met full inclusion criteria. Meta-analysis revealed that overall CDSS uptake was 34.2% (95% CI 23.2 to 47.1%). Uptake was only reported in 12.4% of studies that otherwise met inclusion criteria. Multivariable meta-regression revealed the following factors significantly associated with uptake: (1) formally evaluating the availability and quality of the patient data needed to inform CDSS advice; and (2) identifying and addressing other barriers to the behaviour change targeted by the CDSS.Conclusions and relevanceSystem uptake was seldom reported in CDSS trials. When reported, uptake was low. This represents a major and potentially modifiable barrier to overall CDSS effectiveness. We found that features relating to CDSS context and implementation strategy best predicted uptake. Future studies should measure the impact of addressing these features as part of the CDSS implementation strategy. Uptake reporting must also become standard in future studies reporting CDSS intervention effects.RegistrationPre-registered on PROSPERO, CRD42018092337.
| S-EPMC8908582 | biostudies-literature
Project description:ObjectiveTo identify features of clinical decision support systems critical for improving clinical practice.DesignSystematic review of randomised controlled trials.Data sourcesLiterature searches via Medline, CINAHL, and the Cochrane Controlled Trials Register up to 2003; and searches of reference lists of included studies and relevant reviews.Study selectionStudies had to evaluate the ability of decision support systems to improve clinical practice.Data extractionStudies were assessed for statistically and clinically significant improvement in clinical practice and for the presence of 15 decision support system features whose importance had been repeatedly suggested in the literature.ResultsSeventy studies were included. Decision support systems significantly improved clinical practice in 68% of trials. Univariate analyses revealed that, for five of the system features, interventions possessing the feature were significantly more likely to improve clinical practice than interventions lacking the feature. Multiple logistic regression analysis identified four features as independent predictors of improved clinical practice: automatic provision of decision support as part of clinician workflow (P < 0.00001), provision of recommendations rather than just assessments (P = 0.0187), provision of decision support at the time and location of decision making (P = 0.0263), and computer based decision support (P = 0.0294). Of 32 systems possessing all four features, 30 (94%) significantly improved clinical practice. Furthermore, direct experimental justification was found for providing periodic performance feedback, sharing recommendations with patients, and requesting documentation of reasons for not following recommendations.ConclusionsSeveral features were closely correlated with decision support systems' ability to improve patient care significantly. Clinicians and other stakeholders should implement clinical decision support systems that incorporate these features whenever feasible and appropriate.
| S-EPMC555881 | biostudies-literature
Project description:BACKGROUND:Computerised clinical decision support (CDS) can potentially better inform decisions, and it can help with the management of information overload. It is perceived to be a key component of a learning health care system. Despite its increasing implementation worldwide, it remains uncertain why the effect of CDS varies and which factors make CDS more effective. OBJECTIVE:To examine which factors make CDS strategies more effective on a number of outcomes, including adherence to recommended practice, patient outcome measures, economic measures, provider or patient satisfaction, and medical decision quality. METHODS:We identified randomised controlled trials, non-randomised trials, and controlled before-and-after studies that directly compared CDS implementation with a given factor to CDS without that factor by searching CENTRAL, MEDLINE, EMBASE, and CINAHL and checking reference lists of relevant studies. We considered CDS with any objective for any condition in any healthcare setting. We included CDS interventions that were either displayed on screen or provided on paper and that were directed at healthcare professionals or targeted at both professionals and patients. The reviewers screened the potentially relevant studies in duplicate. They extracted data and assessed risk of bias in independent pairs or individually followed by a double check by another reviewer. We summarised results using medians and interquartile ranges and rated our certainty in the evidence using the GRADE system. RESULTS:We identified 66 head-to-head trials that we synthesised across 14 comparisons of CDS intervention factors. Providing CDS automatically versus on demand led to large improvements in adherence. Displaying CDS on-screen versus on paper led to moderate improvements and making CDS more versus less patient-specific improved adherence modestly. When CDS interventions were combined with professional-oriented strategies, combined with patient-oriented strategies, or combined with staff-oriented strategies, then adherence improved slightly. Providing CDS to patients slightly increased adherence versus CDS aimed at the healthcare provider only. Making CDS advice more explicit and requiring users to respond to the advice made little or no difference. The CDS intervention factors made little or no difference to patient outcomes. The results for economic outcomes and satisfaction outcomes were sparse. CONCLUSION:Multiple factors may affect the success of CDS interventions. CDS may be more effective when the advice is provided automatically and displayed on-screen and when the suggestions are more patient-specific. CDS interventions combined with other strategies probably also improves adherence. Providing CDS directly to patients may also positively affect adherence. The certainty of the evidence was low to moderate for all factors. TRIAL REGISTRATION:PROSPERO, CRD42016033738.
| S-EPMC6102833 | biostudies-literature
Project description:We developed a prototype genomic archiving and communications system to securely store genome data and provide clinical decision support (CDS). This system operates on a client-server model. The client encrypts the data, and the server stores data and performs the computations necessary for CDS. Computations are directly performed on encrypted data, and the client decrypts results. The server cannot decrypt inputs or outputs, which provides strong guarantees of security. We have validated our system with three genomics-based CDS applications. The results demonstrate that it is possible to resolve a long-standing dilemma in genomic data privacy and accessibility, by using a principled cryptographical framework and a mathematical representation of genome data and CDS questions.
| S-EPMC8577277 | biostudies-literature
Project description:BackgroundAutomated clinical decision support systems (CDSS) are associated with improvements in health care delivery to those with long-term conditions, including diabetes. A CDSS was introduced to two Scottish regions (combined diabetes population ~30 000) via a national diabetes electronic health record. This study aims to describe users' reactions to the CDSS and to quantify impact on clinical processes and outcomes over two improvement cycles: December 2013 to February 2014 and August 2014 to November 2014.MethodsFeedback was sought via patient questionnaires, health care professional (HCP) focus groups, and questionnaires. Multivariable regression was used to analyze HCP SCI-Diabetes usage (with respect to CDSS message presence/absence) and case-control comparison of clinical processes/outcomes. Cases were patients whose HCP received a CDSS messages during the study period. Closely matched controls were selected from regions outside the study, following similar clinical practice (without CDSS). Clinical process measures were screening rates for diabetes-related complications. Clinical outcomes included HbA1c at 1 year.ResultsThe CDSS had no adverse impact on consultations. HCPs were generally positive toward CDSS and used it within normal clinical workflow. CDSS messages were generated for 5692 cases, matched to 10 667 controls. Following clinic, the probability of patients being appropriately screened for complications more than doubled for most measures. Mean HbA1c improved in cases and controls but more so in cases (-2.3 mmol/mol [-0.2%] versus -1.1 [-0.1%], P = .003).Discussion and conclusionsThe CDSS was well received; associated with improved efficiencies in working practices; and large improvements in guideline adherence. These evidence-based, early interventions can significantly reduce costly and devastating complications.
| S-EPMC5851216 | biostudies-literature
Project description:BackgroundElectronic medical records are widely used in family practices across Canada and can improve health outcomes. However, recent reports indicate that physicians using electronic medical records work longer and have less direct patient contact which may contribute to burnout. Therefore, new and innovative digital tools are essential to reduce physician workloads and improve patient-physician interaction to address physician burnout. The objective of this study was to assess the efficiency and accuracy of clinical decision-making when using a new preventive care point-of-care clinical decision support system (CDSS). An estimate of the potential annual time savings was also determined. This study also assessed physician reported perceived usefulness and ease of use of the CDSS.MethodsQuantitative and qualitative data were collected during this study. Each participant evaluated two simulated patient charts and identified which preventive care metrics were due. The participants recorded their decisions and the time required to assess each chart. Participants then completed a Technology Acceptance Model survey regarding the perceived usefulness and ease of use of the CDSS, which included qualitative feedback. The amount of time saved was determined and participants' clinical decision-making accuracy was scored against current Canadian preventive care guidelines. The number of preventive care specific visits completed per year was determined using clinic billing data.ResultsThe preventive care CDSS saved an average of 195.7 s of chart review time (249.5 s vs 445.2 s; P < 0.001). A total of 1520 preventive visits were performed at Primrose and Bruyère Family Medicine Centres. Extrapolated across the organization, implementation of the new tool could save 82.6 h per year. Decision-making accuracy was not affected by the new tool (78.4% vs 80.9%, P > 0.05). Participants rated the perceived ease of use and usefulness to be very high.ConclusionsNew digital tools may reduce providers' workload without impacting clinical decision-making accuracy. Participants indicated that the preventive care CDSS was useful and easy to use. Further software development and clinical studies are required to further improve and characterize the effect this new CDSS has when implemented in clinical practice.
| S-EPMC8588582 | biostudies-literature
Project description:BackgroundSpinal disorders are highly prevalent worldwide with high socioeconomic costs. This cost is associated with the demand for treatment and productivity loss, prompting the exploration of technologies to improve patient outcomes. Clinical decision support systems (CDSSs) are computerized systems that are increasingly used to facilitate safe and efficient health care. Their applications range in depth and can be found across health care specialties.ObjectiveThis scoping review aims to explore the use of CDSSs in patients with spinal disorders.MethodsWe used the Joanna Briggs Institute methodological guidance for this scoping review and reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) statement. Databases, including PubMed, Embase, Cochrane, CINAHL, Web of Science, Scopus, ProQuest, and PsycINFO, were searched from inception until October 11, 2022. The included studies examined the use of digitalized CDSSs in patients with spinal disorders.ResultsA total of 4 major CDSS functions were identified from 31 studies: preventing unnecessary imaging (n=8, 26%), aiding diagnosis (n=6, 19%), aiding prognosis (n=11, 35%), and recommending treatment options (n=6, 20%). Most studies used the knowledge-based system. Logistic regression was the most commonly used method, followed by decision tree algorithms. The use of CDSSs to aid in the management of spinal disorders was generally accepted over the threat to physicians' clinical decision-making autonomy.ConclusionsAlthough the effectiveness was frequently evaluated by examining the agreement between the decisions made by the CDSSs and the health care providers, comparing the CDSS recommendations with actual clinical outcomes would be preferable. In addition, future studies on CDSS development should focus on system integration, considering end user's needs and preferences, and external validation and impact studies to assess effectiveness and generalizability.Trial registrationOSF Registries osf.io/dyz3f; https://osf.io/dyz3f.
| S-EPMC10988379 | biostudies-literature
Project description:A clinical decision support system (CDSS) and its components can malfunction due to various reasons. Monitoring the system and detecting its malfunctions can help one to avoid any potential mistakes and associated costs. In this paper, we investigate the problem of detecting changes in the CDSS operation, in particular its monitoring and alerting subsystem, by monitoring its rule firing counts. The detection should be performed online, that is whenever a new datum arrives, we want to have a score indicating how likely there is a change in the system. We develop a new method based on Seasonal-Trend decomposition and likelihood ratio statistics to detect the changes. Experiments on real and simulated data show that our method has a lower delay in detection compared with existing change-point detection methods.
| S-EPMC5546760 | biostudies-other
Project description:Given the multifactorial features characterizing age-related macular degeneration (AMD), the availability of a tool able to provide the individual risk profile is extremely helpful for personalizing the follow-up and treatment protocols of patients. To this purpose, we developed an open-source computational tool named WARE (Wet AMD Risk Evaluation), able to assess the individual risk profile for wet AMD based on genetic and non-genetic factors. In particular, the tool uses genetic risk measures normalized for their relative frequencies in the general population and disease prevalence. WARE is characterized by a user-friendly web page interface that is intended to assist clinicians in reporting risk assessment upon patient evaluation. When using the tool, plots of population risk distribution highlight a “low-risk zone” and a “high-risk zone” into which subjects can fall depending on their risk-assessment result. WARE represents a reliable population-specific computational system for wet AMD risk evaluation that can be exploited to promote preventive actions and personalized medicine approach for affected patients or at-risk individuals. This tool can be suitable to compute the disease risk adjusted to different populations considering their specific genetic factors and related frequencies, non-genetic factors, and the disease prevalence.
| S-EPMC9321802 | biostudies-literature
Project description:ObjectiveTo inform the design of future informatics systems that support the chronic care model.Study designWe describe the development and functionality of a decision support system for the chronic care model of depression treatment, known as collaborative care. Dissemination of evidence-based collaborative care models has been slow, and fidelity to the evidence base has been poor during implementation initiatives. Implementation could be facilitated by a decision support system for depression care managers, the cornerstone of the collaborative care model. The Net Decision Support System (https://www.netdss.net/) is a free Web-based system that was developed to support depression care manager activities and to facilitate the dissemination of collaborative care models that maintain high fidelity to the evidence base.MethodsThe NetDSS was based on intervention materials used for a randomized trial of depression care management that improved clinical outcomes compared with usual care. The NetDSS was developed jointly by a cross-functional design team of psychiatrists, depression care managers, information technology specialists, technical writers, and researchers.ResultsThe NetDSS has the following functional capabilities: patient registry, patient encounter scheduler, trial management, clinical decision support, progress note generator, and workload and outcomes report generator. The NetDSS guides the care manager through a self-documenting patient encounter using evidence-based scripts and self-scoring instruments. The NetDSS has been used to provide evidence-based depression care management to more than 1700 primary care patients.ConclusionIntervention protocols can be successfully converted to Web-based decision support systems that facilitate the implementation of evidence-based chronic care models into routine care with high fidelity.
| S-EPMC3329751 | biostudies-literature