Dataset Information

Evaluation frameworks for digital nursing technologies: analysis, assessment, and guidance. An overview of the literature.

ABSTRACT:

Background

The evaluation of digital nursing technologies (DNT) plays a major role in gaining knowledge about certain aspects of a technology such as acceptance, effectiveness, or efficiency. Evaluation frameworks can help to classify the success or failure of a DNT or to further develop the technology. In general, there are many different evaluation frameworks in the literature that provide overviews of a wide variety of aspects, which makes this a highly diverse field and raises the question how to select a suitable framework. The aim of this article is to provide orientation in the field of comprehensive evaluation frameworks that can be applied to the field of DNT and to conduct a detailed analysis and assessment of these frameworks to guide field researchers.

Methods

This overview was conducted using a three-component search process to identify relevant frameworks. These components were (1) a systematized literature search in PubMed; (2) a narrative review and (3) expert consultations. Data relating to the frameworks' evaluation areas, purpose, perspectives, and success definitions were extracted. Quality criteria were developed in an expert workshop and a strength and weakness assessment was carried out.

Results

Eighteen relevant comprehensive evaluation frameworks for DNT were identified. Nine overarching evaluation areas, seven categories of purposes, five evaluation perspectives and three categories of success definitions could be identified. Eleven quality criteria for the strengths and weaknesses of DNT-related evaluation frameworks were developed and the included frameworks were assessed against them.

Conclusion

Evaluators can use the concise information and quality criteria of this article as a starting point to select and apply appropriate DNT evaluation frameworks for their research projects or to assess the quality of an evaluation framework for DNT, as well as a basis for exploring the questions raised in this article. Future research could address gaps and weaknesses in existing evaluation frameworks, which could improve the quality of future DNT evaluations.

SUBMITTER: Krick T

PROVIDER: S-EPMC8369663 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundHealth economic evaluation of digital nursing technologies (DNT) is important to provide information that helps avoid undesirable developments and implementations as well as increase the chances of success of developed applications. At the same time, studies and evidence on cost-effectiveness are still very rare in this field. Review studies in related technology areas such as telemedicine frequently criticise the quality and comparability of health economic evaluations conducted in this field. Based on a content analysis of methodological literature on the economic evaluation of innovative (digital) technologies in health and nursing, this article aims to identify specific challenges in this research area and offers recommendations on how to address these challenges to promote more sound health economic evaluations in the future.MethodsA rapid review was conducted, consisting of a systematic search in the Pubmed database as well as Google Scholar. In addition, the literature lists of the analysed texts were scoured for additional texts to be included. Methodological literature, single studies, and reviews were included. A total of 536 studies were screened, of which 29 were included in the full text analysis.ResultsBased on the systematic content analysis of the studies under consideration, 10 specific methodological challenges are identified, and the methodological recommendations were examined for consideration. A particular focus was given to whether specific methodological approaches might be needed in the context of evaluating the efficiency of DNT.ConclusionMany of the challenges identified for the health economic evaluations of digital nursing technologies are comparable to those of other complex health care interventions. The recommendations discussed can help to alleviate those challenges. Future research should focus on alternative approaches to assessing causality in different phases of technology development while maintaining high evidence standards. High-evidence economic assessment of technologies in nursing care should be carried out in routine use, especially if they are intended to be reimbursed by the social insurance.

Project description:BackgroundDigital health technologies (DHTs) generate a large volume of information used in health care for administrative, educational, research, and clinical purposes. The clinical use of digital information for diagnostic, therapeutic, and prognostic purposes has multiple patient safety problems, some of which result from poor information quality (IQ).ObjectiveThis systematic review aims to synthesize an IQ framework that could be used to evaluate the extent to which digital health information is fit for clinical purposes.MethodsThe review was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines. We searched Embase, MEDLINE, PubMed, CINAHL, Maternity and Infant Care, PsycINFO, Global Health, ProQuest Dissertations and Theses Global, Scopus, and HMIC (the Health Management Information Consortium) from inception until October 2019. Multidimensional IQ frameworks for assessing DHTs used in the clinical context by health care professionals were included. A thematic synthesis approach was used to synthesize the Clinical Information Quality (CLIQ) framework for digital health.ResultsWe identified 10 existing IQ frameworks from which we developed the CLIQ framework for digital health with 13 unique dimensions: accessibility, completeness, portability, security, timeliness, accuracy, interpretability, plausibility, provenance, relevance, conformance, consistency, and maintainability, which were categorized into 3 meaningful categories: availability, informativeness, and usability.ConclusionsThis systematic review highlights the importance of the IQ of DHTs and its relevance to patient safety. The CLIQ framework for digital health will be useful in evaluating and conceptualizing IQ issues associated with digital health, thus forestalling potential patient safety problems.Trial registrationPROSPERO International Prospective Register of Systematic Reviews CRD42018097142; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97142.International registered report identifier (irrid)RR2-10.1136/bmjopen-2018-024722.

Project description:BackgroundDigital nursing technologies (DNT) comprise an expanding, highly diverse field of research, explored using a wide variety of methods and tools. Study results are therefore difficult to compare, which raises the question how effectiveness of DNT can be adequately measured. Methods currently used might not be sufficient for certain specific nursing contexts. A comprehensive outcome framework that shows the multitude of possible outcome areas could be useful to generate more comparable results. The aim of the present study is to develop an outcome framework for DNT and to indicate which outcome areas have been most frequently evaluated in previous studies and how this has been done.MethodsWe combined an inductive and deductive approach to develop the framework. The numerical analysis is based on a scoping review focussing on the effectiveness of DNT for persons in need of care, formal or informal caregivers or care institutions. Nine databases were included in the screening: Medline, Scopus, CINAHL, Cochrane Library, ACM Digital Library, IEEE Xplore, the Collection of Computer Science Bibliographies, GeroLit and CareLit. Additional literature searches and expert interviews were included.ResultsThe developed framework comprises four outcome target groups and 47 outcome areas. There are considerable differences in the researched outcome areas for the individual outcome target groups. Persons in need of care were by far the most frequently surveyed, particularly with respect to their psychological health. There are much fewer studies on formal and informal caregivers, and it is particularly noticeable that the quality of life of both groups has rarely been investigated. Care process quality was most frequently researched for organisations.ConclusionWe were able to provide a comprehensive DNT outcome framework, thereby identifying the outcome tools used and the less researched outcome areas. We recommend a detailed investigation of all areas and tools in future research projects with a view to initiating a discussion on the differing importance of existing outcome areas and on a standardisation of outcome tools. We also recommend the development of outcome areas for the macro level of effectiveness assessment.

Project description:BACKGROUND:Information and communication technologies (ICTs) are becoming an impetus for quality health care delivery by nurses. The use of ICTs by nurses can impact their practice, modifying the ways in which they plan, provide, document, and review clinical care. OBJECTIVE:An overview of systematic reviews was conducted to develop a broad picture of the dimensions and indicators of nursing care that have the potential to be influenced by the use of ICTs. METHODS:Quantitative, mixed-method, and qualitative reviews that aimed to evaluate the influence of four eHealth domains (eg, management, computerized decision support systems [CDSSs], communication, and information systems) on nursing care were included. We used the nursing care performance framework (NCPF) as an extraction grid and analytical tool. This model illustrates how the interplay between nursing resources and the nursing services can produce changes in patient conditions. The primary outcomes included nurses' practice environment, nursing processes, professional satisfaction, and nursing-sensitive outcomes. The secondary outcomes included satisfaction or dissatisfaction with ICTs according to nurses' and patients' perspectives. Reviews published in English, French, or Spanish from January 1, 1995 to January 15, 2015, were considered. RESULTS:A total of 5515 titles or abstracts were assessed for eligibility and full-text papers of 72 articles were retrieved for detailed evaluation. It was found that 22 reviews published between 2002 and 2015 met the eligibility criteria. Many nursing care themes (ie, indicators) were influenced by the use of ICTs, including time management; time spent on patient care; documentation time; information quality and access; quality of documentation; knowledge updating and utilization; nurse autonomy; intra and interprofessional collaboration; nurses' competencies and skills; nurse-patient relationship; assessment, care planning, and evaluation; teaching of patients and families; communication and care coordination; perspectives of the quality of care provided; nurses and patients satisfaction or dissatisfaction with ICTs; patient comfort and quality of life related to care; empowerment; and functional status. CONCLUSIONS:The findings led to the identification of 19 indicators related to nursing care that are impacted by the use of ICTs. To the best of our knowledge, this was the first attempt to apply NCPF in the ICTs' context. This broad representation could be kept in mind when it will be the time to plan and to implement emerging ICTs in health care settings. TRIAL REGISTRATION:PROSPERO International Prospective Register of Systematic Reviews: CRD42014014762; http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014014762 (Archived by WebCite at http://www.webcitation.org/6pIhMLBZh).

Project description:BackgroundDigital health technologies (DHTs), such as electronic health records and prescribing systems, are transforming health care delivery around the world. The quality of information in DHTs is key to the quality and safety of care. We developed a novel clinical information quality (CLIQ) framework to assess the quality of clinical information in DHTs.ObjectiveThis study explored clinicians' perspectives on the relevance, definition, and assessment of information quality dimensions in the CLIQ framework.MethodsWe used a systematic and iterative eDelphi approach to engage clinicians who had information governance roles or personal interest in information governance; the clinicians were recruited through purposive and snowball sampling techniques. Data were collected using semistructured online questionnaires until consensus was reached on the information quality dimensions in the CLIQ framework. Responses on the relevance of the dimensions were summarized to inform decisions on retention of the dimensions according to prespecified rules. Thematic analysis of the free-text responses was used to revise definitions and the assessment of dimensions.ResultsThirty-five clinicians from 10 countries participated in the study, which was concluded after the second round. Consensus was reached on all dimensions and categories in the CLIQ framework: informativeness (accuracy, completeness, interpretability, plausibility, provenance, and relevance), availability (accessibility, portability, security, and timeliness), and usability (conformance, consistency, and maintainability). A new dimension, searchability, was introduced in the availability category to account for the ease of finding needed information in the DHTs. Certain dimensions were renamed, and some definitions were rephrased to improve clarity.ConclusionsThe CLIQ framework reached a high expert consensus and clarity of language relating to the information quality dimensions. The framework can be used by health care managers and institutions as a pragmatic tool for identifying and forestalling information quality problems that could compromise patient safety and quality of care.International registered report identifier (irrid)RR2-10.1136/bmjopen-2021-057430.

Project description:BackgroundDigital contact tracing apps have the potential to augment contact tracing systems and disrupt COVID-19 transmission by rapidly identifying secondary cases prior to the onset of infectiousness and linking them into a system of quarantine, testing, and health care worker case management. The international experience of digital contact tracing apps during the COVID-19 pandemic demonstrates how challenging their design and deployment are.ObjectiveThis study aims to derive and summarize best practice guidance for the design of the ideal digital contact tracing app.MethodsA collaborative cross-disciplinary approach was used to derive best practice guidance for designing the ideal digital contact tracing app. A search of the indexed and gray literature was conducted to identify articles describing or evaluating digital contact tracing apps. MEDLINE was searched using a combination of free-text terms and Medical Subject Headings search terms. Gray literature sources searched were the World Health Organization Institutional Repository for Information Sharing, the European Centre for Disease Prevention and Control publications library, and Google, including the websites of many health protection authorities. Articles that were acceptable for inclusion in this evidence synthesis were peer-reviewed publications, cohort studies, randomized trials, modeling studies, technical reports, white papers, and media reports related to digital contact tracing.ResultsEthical, user experience, privacy and data protection, technical, clinical and societal, and evaluation considerations were identified from the literature. The ideal digital contact tracing app should be voluntary and should be equitably available and accessible. User engagement could be enhanced by small financial incentives, enabling users to tailor aspects of the app to their particular needs and integrating digital contact tracing apps into the wider public health information campaign. Adherence to the principles of good data protection and privacy by design is important to convince target populations to download and use digital contact tracing apps. Bluetooth Low Energy is recommended for a digital contact tracing app's contact event detection, but combining it with ultrasound technology may improve a digital contact tracing app's accuracy. A decentralized privacy-preserving protocol should be followed to enable digital contact tracing app users to exchange and record temporary contact numbers during contact events. The ideal digital contact tracing app should define and risk-stratify contact events according to proximity, duration of contact, and the infectiousness of the case at the time of contact. Evaluating digital contact tracing apps requires data to quantify app downloads, use among COVID-19 cases, successful contact alert generation, contact alert receivers, contact alert receivers that adhere to quarantine and testing recommendations, and the number of contact alert receivers who subsequently are tested positive for COVID-19. The outcomes of digital contact tracing apps' evaluations should be openly reported to allow for the wider public to review the evaluation of the app.ConclusionsIn conclusion, key considerations and best practice guidance for the design of the ideal digital contact tracing app were derived from the literature.