Dataset Information

Evaluation of a clinical decision support system for rare diseases: a qualitative study.

ABSTRACT:

Background

Rare Diseases (RDs) are difficult to diagnose. Clinical Decision Support Systems (CDSS) could support the diagnosis for RDs. The Medical Informatics in Research and Medicine (MIRACUM) consortium developed a CDSS for RDs based on distributed clinical data from eight German university hospitals. To support the diagnosis for difficult patient cases, the CDSS uses data from the different hospitals to perform a patient similarity analysis to obtain an indication of a diagnosis. To optimize our CDSS, we conducted a qualitative study to investigate usability and functionality of our designed CDSS.

Methods

We performed a Thinking Aloud Test (TA-Test) with RDs experts working in Rare Diseases Centers (RDCs) at MIRACUM locations which are specialized in diagnosis and treatment of RDs. An instruction sheet with tasks was prepared that the participants should perform with the CDSS during the study. The TA-Test was recorded on audio and video, whereas the resulting transcripts were analysed with a qualitative content analysis, as a ruled-guided fixed procedure to analyse text-based data. Furthermore, a questionnaire was handed out at the end of the study including the System Usability Scale (SUS).

Results

A total of eight experts from eight MIRACUM locations with an established RDC were included in the study. Results indicate that more detailed information about patients, such as descriptive attributes or findings, can help the system perform better. The system was rated positively in terms of functionality, such as functions that enable the user to obtain an overview of similar patients or medical history of a patient. However, there is a lack of transparency in the results of the CDSS patient similarity analysis. The study participants often stated that the system should present the user with an overview of exact symptoms, diagnosis, and other characteristics that define two patients as similar. In the usability section, the CDSS received a score of 73.21 points, which is ranked as good usability.

Conclusions

This qualitative study investigated the usability and functionality of a CDSS of RDs. Despite positive feedback about functionality of system, the CDSS still requires some revisions and improvement in transparency of the patient similarity analysis.

SUBMITTER: Schaaf J

PROVIDER: S-EPMC7890997 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:Patients with rare diseases (RDs) are often diagnosed too late or not at all. Clinical decision support systems (CDSSs) could support the diagnosis in RDs. The MIRACUM (Medical Informatics in Research and Medicine) consortium, which is one of four funded consortia in the German Medical Informatics Initiative, will develop a CDSS for RDs based on distributed clinical data from ten university hospitals. This qualitative study aims to investigate (1) the relevant organizational conditions for the operation of a CDSS for RDs when diagnose patients (e.g. the diagnosis workflow), (2) which data is necessary for decision support, and (3) the appropriate user group for such a CDSS. METHODS:Interviews were carried out with RDs experts. Participants were recruited from staff physicians at the Rare Disease Centers (RDCs) at the MIRACUM locations, which offer diagnosis and treatment of RDs. An interview guide was developed with a category-guided deductive approach. The interviews were recorded on an audio device and then transcribed into written form. We continued data collection until all interviews were completed. Afterwards, data analysis was performed using Mayring's qualitative content analysis approach. RESULTS:A total of seven experts were included in the study. The results show that medical center guides and physicians from RDC B-centers (with a focus on different RDs) are involved in the diagnostic process. Furthermore, interdisciplinary case discussions between physicians are conducted. The experts explained that RDs exist which cannot be fully differentiated, but rather described only by their overall symptoms or findings: diagnosis is dependent on the disease or disease group. At the end of the diagnostic process, most centers prepare a summary of the patient case. Furthermore, the experts considered both physicians and experts from the B-centers to be potential users of a CDSS. The experts also have different experiences with CDSS for RDs. CONCLUSIONS:This qualitative study is a first step towards establishing the requirements for the development of a CDSS for RDs. Further research is necessary to create solutions by also including the experts on RDs.

Project description:BackgroundRare Diseases (RDs), which are defined as diseases affecting no more than 5 out of 10,000 people, are often severe, chronic and life-threatening. A main problem is the delay in diagnosing RDs. Clinical decision support systems (CDSSs) for RDs are software systems to support clinicians in the diagnosis of patients with RDs. Due to their clinical importance, we conducted a scoping review to determine which CDSSs are available to support the diagnosis of RDs patients, whether the CDSSs are available to be used by clinicians and which functionalities and data are used to provide decision support.MethodsWe searched PubMed for CDSSs in RDs published between December 16, 2008 and December 16, 2018. Only English articles, original peer reviewed journals and conference papers describing a clinical prototype or a routine use of CDSSs were included. For data charting, we used the data items "Objective and background of the publication/project", "System or project name", "Functionality", "Type of clinical data", "Rare Diseases covered", "Development status", "System availability", "Data entry and integration", "Last software update" and "Clinical usage".ResultsThe search identified 636 articles. After title and abstracting screening, as well as assessing the eligibility criteria for full-text screening, 22 articles describing 19 different CDSSs were identified. Three types of CDSSs were classified: "Analysis or comparison of genetic and phenotypic data," "machine learning" and "information retrieval". Twelve of nineteen CDSSs use phenotypic and genetic data, followed by clinical data, literature databases and patient questionnaires. Fourteen of nineteen CDSSs are fully developed systems and therefore publicly available. Data can be entered or uploaded manually in six CDSSs, whereas for four CDSSs no information for data integration was available. Only seven CDSSs allow further ways of data integration. thirteen CDSS do not provide information about clinical usage.ConclusionsDifferent CDSS for various purposes are available, yet clinicians have to determine which is best for their patient. To allow a more precise usage, future research has to focus on CDSSs RDs data integration, clinical usage and updating clinical knowledge. It remains interesting which of the CDSSs will be used and maintained in the future.

Project description:BackgroundThe quality and safety of prescribing in general practice is important, Clinical decision support (CDS) systems can be used which present alerts to health professionals when prescribing in order to identify patients at risk of potentially hazardous prescribing. It is known that such computerised alerts may improve the safety of prescribing in hospitals but their implementation and sustainable use in general practice is less well understood. We aimed to understand the factors that influenced the successful implementation and sustained use in primary care of a CDS system.MethodsParticipants were purposively recruited from Clinical Commissioning Groups (CCGs) and general practices in the North West and East Midlands regions of England and from the CDS developers. We conducted face-to-face and telephone-based semi-structured qualitative interviews with staff stakeholders. A selection of participants was interviewed longitudinally to explore the further sustainability 1-2 years after implementation of the CDS system. The analysis, informed by Normalisation Process Theory (NPT), was thematic, iterative and conducted alongside data collection.ResultsThirty-nine interviews were conducted either individually or in groups, with 33 stakeholders, including 11 follow-up interviews. Eight themes were interpreted in alignment with the four NPT constructs: Coherence (The purpose of the CDS: Enhancing medication safety and improving cost effectiveness; Relationship of users to the technology; Engagement and communication between different stakeholders); Cognitive Participation (Management of the profile of alerts); Collective Action (Prescribing in general practice, patient and population characteristics and engagement with patients; Knowledge);and Reflexive Monitoring (Sustaining the use of the CDS through maintenance and customisation; Learning and behaviour change. Participants saw that the CDS could have a role in enhancing medication safety and in the quality of care. Engagement through communication and support for local primary care providers and management leaders was considered important for successful implementation. Management of prescribing alert profiles for general practices was a dynamic process evolving over time. At regional management levels, work was required to adapt, and modify the system to optimise its use in practice and fulfil local priorities. Contextual factors, including patient and population characteristics, could impact upon the decision-making processes of prescribers influencing the response to alerts. The CDS could operate as a knowledge base allowing prescribers access to evidence-based information that they otherwise would not have.ConclusionsThis qualitative evaluation utilised NPT to understand the implementation, use and sustainability of a widely deployed CDS system offering prescribing alerts in general practice. The system was understood as having a role in medication safety in providing relevant patient specific information to prescribers in a timely manner. Engagement between stakeholders was considered important for the intervention in ensuring prescribers continued to utilise its functionality. Sustained implementation might be enhanced by careful profile management of the suite of alerts in the system. Our findings suggest that the use and sustainability of the CDS was related to prescribers' perceptions of the relevance of alerts. Shared understanding of the purpose of the CDS between CCGS and general practices particularly in balancing cost saving and safety messages could be beneficial.

Project description:IntroductionThe burden of mental health-related visits to emergency departments (EDs) is growing, and agitation episodes are prevalent with such visits. Best practice guidance from experts recommends early assessment of at-risk populations and pre-emptive intervention using de-escalation techniques to prevent agitation. Time pressure, fluctuating work demands, and other systems-related factors pose challenges to efficient decision-making and adoption of best practice recommendations during an unfolding behavioural crisis. As such, we propose to design, develop and evaluate a computerised clinical decision support (CDS) system, Early Detection and Treatment to Reduce Events with Agitation Tool (ED-TREAT). We aim to identify patients at risk of agitation and guide ED clinicians through appropriate risk assessment and timely interventions to prevent agitation with a goal of minimising restraint use and improving patient experience and outcomes.Methods and analysisThis study describes the formative evaluation of the health record embedded CDS tool. Under aim 1, the study will collect qualitative data to design and develop ED-TREAT using a contextual design approach and an iterative user-centred design process. Participants will include potential CDS users, that is, ED physicians, nurses, technicians, as well as patients with lived experience of restraint use for behavioural crisis management during an ED visit. We will use purposive sampling to ensure the full spectrum of perspectives until we reach thematic saturation. Next, under aim 2, the study will conduct a pilot, randomised controlled trial of ED-TREAT at two adult ED sites in a regional health system in the Northeast USA to evaluate the feasibility, fidelity and bedside acceptability of ED-TREAT. We aim to recruit a total of at least 26 eligible subjects under the pilot trial.Ethics and disseminationEthical approval by the Yale University Human Investigation Committee was obtained in 2021 (HIC# 2000030893 and 2000030906). All participants will provide informed verbal consent prior to being enrolled in the study. Results will be disseminated through publications in open-access, peer-reviewed journals, via scientific presentations or through direct email notifications.Trial registration numberNCT04959279; Pre-results.