Dataset Information

An AI-Powered Clinical Decision Support System to Predict Flares in Rheumatoid Arthritis: A Pilot Study.

ABSTRACT: Treat-to-target (T2T) is a main therapeutic strategy in rheumatology; however, patients and rheumatologists currently have little support in making the best treatment decision. Clinical decision support systems (CDSSs) could offer this support. The aim of this study was to investigate the accuracy, effectiveness, usability, and acceptance of such a CDSS-Rheuma Care Manager (RCM)-including an artificial intelligence (AI)-powered flare risk prediction tool to support the management of rheumatoid arthritis (RA). Longitudinal clinical routine data of RA patients were used to develop and test the RCM. Based on ten real-world patient vignettes, five physicians were asked to assess patients' flare risk, provide a treatment decision, and assess their decision confidence without and with access to the RCM for predicting flare risk. RCM usability and acceptance were assessed using the system usability scale (SUS) and net promoter score (NPS). The flare prediction tool reached a sensitivity of 72%, a specificity of 76%, and an AUROC of 0.80. Perceived flare risk and treatment decisions varied largely between physicians. Having access to the flare risk prediction feature numerically increased decision confidence (3.5/5 to 3.7/5), reduced deviations between physicians and the prediction tool (20% to 12% for half dosage flare prediction), and resulted in more treatment reductions (42% to 50% vs. 20%). RCM usability (SUS) was rated as good (82/100) and was well accepted (mean NPS score 7/10). CDSS usage could support physicians by decreasing assessment deviations and increasing treatment decision confidence.

SUBMITTER: Labinsky H

PROVIDER: S-EPMC9818406 | biostudies-literature | 2023 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An AI-Powered Clinical Decision Support System to Predict Flares in Rheumatoid Arthritis: A Pilot Study.

Labinsky Hannah H Ukalovic Dubravka D Hartmann Fabian F Runft Vanessa V Wichmann André A Jakubcik Jan J Gambel Kira K Otani Katharina K Morf Harriet H Taubmann Jule J Fagni Filippo F Kleyer Arnd A Simon David D Schett Georg G Reichert Matthias M Knitza Johannes J

Diagnostics (Basel, Switzerland) 20230101 1

Treat-to-target (T2T) is a main therapeutic strategy in rheumatology; however, patients and rheumatologists currently have little support in making the best treatment decision. Clinical decision support systems (CDSSs) could offer this support. The aim of this study was to investigate the accuracy, effectiveness, usability, and acceptance of such a CDSS-Rheuma Care Manager (RCM)-including an artificial intelligence (AI)-powered flare risk prediction tool to support the management of rheumatoid a ...[more]

PMID: 36611439

Similar Datasets

Project description:Involvement of many variables, uncertainty in treatment response, and inter-patient heterogeneity challenge objective decision-making in dynamic treatment regime (DTR) in oncology. Advanced machine learning analytics in conjunction with information-rich dense multi-omics data have the ability to overcome such challenges. We have developed a comprehensive artificial intelligence (AI)-based optimal decision-making framework for assisting oncologists in DTR. In this work, we demonstrate the proposed framework to Knowledge Based Response-Adaptive Radiotherapy (KBR-ART) applications by developing an interactive software tool entitled Adaptive Radiotherapy Clinical Decision Support (ARCliDS). ARCliDS is composed of two main components: Artifcial RT Environment (ARTE) and Optimal Decision Maker (ODM). ARTE is designed as a Markov decision process and modeled via supervised learning. Given a patient's pre- and during-treatment information, ARTE can estimate treatment outcomes for a selected daily dosage value (radiation fraction size). ODM is formulated using reinforcement learning and is trained on ARTE. ODM can recommend optimal daily dosage adjustments to maximize the tumor local control probability and minimize the side effects. Graph Neural Networks (GNN) are applied to exploit the inter-feature relationships for improved modeling performance and a novel double GNN architecture is designed to avoid nonphysical treatment response. Datasets of size 117 and 292 were available from two clinical trials on adaptive RT in non-small cell lung cancer (NSCLC) patients and adaptive stereotactic body RT (SBRT) in hepatocellular carcinoma (HCC) patients, respectively. For training and validation, dense data with 297 features were available for 67 NSCLC patients and 110 features for 71 HCC patients. To increase the sample size for ODM training, we applied Generative Adversarial Networks to generate 10,000 synthetic patients. The ODM was trained on the synthetic patients and validated on the original dataset. We found that, Double GNN architecture was able to correct the nonphysical dose-response trend and improve ARCliDS recommendation. The average root mean squared difference (RMSD) between ARCliDS recommendation and reported clinical decisions using double GNNs were 0.61 [0.03] Gy/frac (mean [sem]) for adaptive RT in NSCLC patients and 2.96 [0.42] Gy/frac for adaptive SBRT HCC compared to the single GNN's RMSDs of 0.97 [0.12] Gy/frac and 4.75 [0.16] Gy/frac, respectively. Overall, For NSCLC and HCC, ARCliDS with double GNNs was able to reproduce 36% and 50% of the good clinical decisions (local control and no side effects) and improve 74% and 30% of the bad clinical decisions, respectively. In conclusion, ARCliDS is the first web-based software dedicated to assist KBR-ART with multi-omics data. ARCliDS can learn from the reported clinical decisions and facilitate AI-assisted clinical decision-making for improving the outcomes in DTR.

Project description:Patients with chronic kidney disease (CKD) necessitate specialized renal diets to prevent complications such as hyperkalemia and hyperphosphatemia. A comprehensive assessment of food components is pivotal, yet burdensome for healthcare providers. With evolving artificial intelligence (AI) technology, models such as ChatGPT, Bard AI, and Bing Chat can be instrumental in educating patients and assisting professionals. To gauge the efficacy of different AI models in discerning potassium and phosphorus content in foods, four AI models-ChatGPT 3.5, ChatGPT 4, Bard AI, and Bing Chat-were evaluated. A total of 240 food items, curated from the Mayo Clinic Renal Diet Handbook for CKD patients, were input into each model. These items were characterized by their potassium (149 items) and phosphorus (91 items) content. Each model was tasked to categorize the items into high or low potassium and high phosphorus content. The results were juxtaposed with the Mayo Clinic Renal Diet Handbook's recommendations. The concordance between repeated sessions was also evaluated to assess model consistency. Among the models tested, ChatGPT 4 displayed superior performance in identifying potassium content, correctly classifying 81% of the foods. It accurately discerned 60% of low potassium and 99% of high potassium foods. In comparison, ChatGPT 3.5 exhibited a 66% accuracy rate. Bard AI and Bing Chat models had an accuracy rate of 79% and 81%, respectively. Regarding phosphorus content, Bard AI stood out with a flawless 100% accuracy rate. ChatGPT 3.5 and Bing Chat recognized 85% and 89% of the high phosphorus foods correctly, while ChatGPT 4 registered a 77% accuracy rate. Emerging AI models manifest a diverse range of accuracy in discerning potassium and phosphorus content in foods suitable for CKD patients. ChatGPT 4, in particular, showed a marked improvement over its predecessor, especially in detecting potassium content. The Bard AI model exhibited exceptional precision for phosphorus identification. This study underscores the potential of AI models as efficient tools in renal dietary planning, though refinements are warranted for optimal utility.

Project description:BackgroundGiven the opportunities created by artificial intelligence (AI) based decision support systems in healthcare, the vital question is whether clinicians are willing to use this technology as an integral part of clinical workflow.PurposeThis study leverages validated questions to formulate an online survey and consequently explore cognitive human factors influencing clinicians' intention to use an AI-based Blood Utilization Calculator (BUC), an AI system embedded in the electronic health record that delivers data-driven personalized recommendations for the number of packed red blood cells to transfuse for a given patient.MethodA purposeful sampling strategy was used to exclusively include BUC users who are clinicians in a university hospital in Wisconsin. We recruited 119 BUC users who completed the entire survey. We leveraged structural equation modeling to capture the direct and indirect effects of "AI Perception" and "Expectancy" on clinicians' Intention to use the technology when mediated by "Perceived Risk".ResultsThe findings indicate a significant negative relationship concerning the direct impact of AI's perception on BUC Risk (ß = -0.23, p < 0.001). Similarly, Expectancy had a significant negative effect on Risk (ß = -0.49, p < 0.001). We also noted a significant negative impact of Risk on the Intent to use BUC (ß = -0.34, p < 0.001). Regarding the indirect effect of Expectancy on the Intent to Use BUC, the findings show a significant positive impact mediated by Risk (ß = 0.17, p = 0.004). The study noted a significant positive and indirect effect of AI Perception on the Intent to Use BUC when mediated by risk (ß = 0.08, p = 0.027). Overall, this study demonstrated the influences of expectancy, perceived risk, and perception of AI on clinicians' intent to use BUC (an AI system). AI developers need to emphasize the benefits of AI technology, ensure ease of use (effort expectancy), clarify the system's potential (performance expectancy), and minimize the risk perceptions by improving the overall design.ConclusionIdentifying the factors that determine clinicians' intent to use AI-based decision support systems can help improve technology adoption and use in the healthcare domain. Enhanced and safe adoption of AI can uplift the overall care process and help standardize clinical decisions and procedures. An improved AI adoption in healthcare will help clinicians share their everyday clinical workload and make critical decisions.

Project description:BackgroundRA flares are common and disabling. They are described in terms of worsening inflammation but pain and inflammation are often discordant. To inform treatment decisions, we investigated whether inflammatory and pain flares are discrete entities.MethodsPeople from the Early RA Network (ERAN) cohort were assessed annually up to 11 years after presentation (n = 719, 3703 person-years of follow up). Flare events were defined in 2 different ways that were analysed in parallel; DAS28 or Pain Flares. DAS28 Flares satisfied OMERACT flare criteria of increases in DAS28 since the previous assessment (≥1.2 points if active RA or ≥ 0.6 points if inactive RA). A ≥ 4.8-point worsening of SF36-Bodily Pain score defined Pain Flares. The first documented episode of each of DAS28 and Pain Flare in each person was analysed. Subgroups within DAS28 and Pain Flares were determined using Latent Class Analysis. Clinical course was compared between flare subgroups.ResultsDAS28 (45%) and Pain Flares (52%) were each common but usually discordant, with 60% of participants in DAS28 Flare not concurrently in Pain Flare, and 64% of those in Pain Flare not concurrently in DAS28 Flare. Three discrete DAS28 Flare subgroups were identified. One was characterised by increases in tender/swollen joint counts (14.4%), a second by increases in symptoms (13.1%), and a third displayed lower flare severity (72.5%). Two discrete Pain Flare subgroups were identified. One occurred following low disease activity and symptoms (88.6%), and the other occurred on the background of ongoing active disease and pain (11.4%). Despite the observed differences between DAS28 and Pain Flares, each was associated with increased disability which persisted beyond the flare episode.ConclusionFlares are both common and heterogeneous in people with RA. Furthermore our findings indicate that for some patients there is a discordance between inflammation and pain in flare events. This discrete flare subgroups might reflect different underlying inflammation and pain mechanisms. Treatments addressing different mechanisms might be required to reduce persistent disability after DAS28 and Pain Flares.

Project description:BackgroundRheumatoid arthritis flares have a profound effect on patients, causing pain and disability. However, flares often occur between regularly scheduled health care provider visits and are, therefore, difficult to monitor and manage. We sought to develop a mobile phone app combined with a population management system to help track RA flares between visits.ObjectiveThe objective of this study is to implement the mobile app plus the population management system to monitor rheumatoid arthritis disease activity between scheduled health care provider visits over a period of 6 months.MethodsThis is a randomized controlled trial that lasts for 6 months for each participant. We aim to recruit 190 patients, randomized 50:50 to the intervention group versus the control group. The intervention group will be assigned the mobile app and be prompted to answer daily questionnaires sent to their mobile devices. Both groups will be assigned a population manager, who will communicate with the participants via telephone at 6 weeks and 18 weeks. The population manager will also communicate with the participants in the intervention group if their responses indicate a sustained increase in rheumatoid arthritis disease activity. To assess patient satisfaction, the primary outcomes will be scores on the Treatment Satisfaction Questionnaire for Medication as well as the Perceived Efficacy in Patient-Physician Interactions questionnaire at 6 months. To determine the effect of the mobile app on rheumatoid arthritis disease activity, the primary outcome will be the Clinical Disease Activity Index at 6 months.ResultsThe trial started in November 2016, and an estimated 2.5 years will be necessary to complete the study. Study results are expected to be published by the end of 2019.ConclusionsThe completion of this study will provide important data regarding the following: (1) the assessment of validated outcome measures to assess rheumatoid arthritis disease activity with a mobile app between routinely scheduled health care provider visits, (2) patient engagement in monitoring their condition, and (3) communication between patients and health care providers through the population management system.Trial registrationClinicalTrials.gov NCT02822521, http://clinicaltrials.gov/ct2/show/NCT02822521 (Archived by WebCite at http://www.webcitation.org/6xed3kGPd).

Project description:ImportanceArtificial intelligence (AI) has gained considerable attention in health care, yet concerns have been raised around appropriate methods and fairness. Current AI reporting guidelines do not provide a means of quantifying overall quality of AI research, limiting their ability to compare models addressing the same clinical question.ObjectiveTo develop a tool (APPRAISE-AI) to evaluate the methodological and reporting quality of AI prediction models for clinical decision support.Design, setting, and participantsThis quality improvement study evaluated AI studies in the model development, silent, and clinical trial phases using the APPRAISE-AI tool, a quantitative method for evaluating quality of AI studies across 6 domains: clinical relevance, data quality, methodological conduct, robustness of results, reporting quality, and reproducibility. These domains included 24 items with a maximum overall score of 100 points. Points were assigned to each item, with higher points indicating stronger methodological or reporting quality. The tool was applied to a systematic review on machine learning to estimate sepsis that included articles published until September 13, 2019. Data analysis was performed from September to December 2022.Main outcomes and measuresThe primary outcomes were interrater and intrarater reliability and the correlation between APPRAISE-AI scores and expert scores, 3-year citation rate, number of Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) low risk-of-bias domains, and overall adherence to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement.ResultsA total of 28 studies were included. Overall APPRAISE-AI scores ranged from 33 (low quality) to 67 (high quality). Most studies were moderate quality. The 5 lowest scoring items included source of data, sample size calculation, bias assessment, error analysis, and transparency. Overall APPRAISE-AI scores were associated with expert scores (Spearman ρ, 0.82; 95% CI, 0.64-0.91; P < .001), 3-year citation rate (Spearman ρ, 0.69; 95% CI, 0.43-0.85; P < .001), number of QUADAS-2 low risk-of-bias domains (Spearman ρ, 0.56; 95% CI, 0.24-0.77; P = .002), and adherence to the TRIPOD statement (Spearman ρ, 0.87; 95% CI, 0.73-0.94; P < .001). Intraclass correlation coefficient ranges for interrater and intrarater reliability were 0.74 to 1.00 for individual items, 0.81 to 0.99 for individual domains, and 0.91 to 0.98 for overall scores.Conclusions and relevanceIn this quality improvement study, APPRAISE-AI demonstrated strong interrater and intrarater reliability and correlated well with several study quality measures. This tool may provide a quantitative approach for investigators, reviewers, editors, and funding organizations to compare the research quality across AI studies for clinical decision support.

Dataset Information

An AI-Powered Clinical Decision Support System to Predict Flares in Rheumatoid Arthritis: A Pilot Study.

Publications

An AI-Powered Clinical Decision Support System to Predict Flares in Rheumatoid Arthritis: A Pilot Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets