Dataset Information

Evidence-Based Evaluation of eHealth Interventions: Systematic Literature Review.

ABSTRACT:

Background

Until now, the use of technology in health care was driven mostly by the assumptions about the benefits of electronic health (eHealth) rather than its evidence. It is noticeable that the magnitude of evidence of effectiveness and efficiency of eHealth is not proportionate to the number of interventions that are regularly conducted. Reliable evidence generated through comprehensive evaluation of eHealth interventions may accelerate the growth of eHealth for long-term successful implementation and help to experience eHealth benefits in an enhanced way.

Objective

This study aimed to understand how the evidence of effectiveness and efficiency of eHealth can be generated through evaluation. Hence, we aim to discern (1) how evaluation is conducted in distinct eHealth intervention phases, (2) the aspects of effectiveness and efficiency that are typically evaluated during eHealth interventions, and (3) how eHealth interventions are evaluated in practice.

Methods

A systematic literature review was conducted to explore the evaluation methods for eHealth interventions. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines were followed. We searched Google Scholar and Scopus for the published papers that addressed the evaluation of eHealth or described an eHealth intervention study. A qualitative analysis of the selected papers was conducted in several steps.

Results

We intended to see how the process of evaluation unfolds in distinct phases of an eHealth intervention. We revealed that in practice and in several conceptual papers, evaluation is performed at the end of the intervention. There are some studies that discuss the importance of conducting evaluation throughout the intervention; however, in practice, we found no case study that followed this. For our second research question, we discovered aspects of efficiency and effectiveness that are proposed to be assessed during interventions. The aspects that were recurrent in the conceptual papers include clinical, human and social, organizational, technological, cost, ethical and legal, and transferability. However, the case studies reviewed only evaluate the clinical and human and social aspects. At the end of the paper, we discussed a novel approach to look into the evaluation. Our intention was to stir up a discussion around this approach with the hope that it might be able to gather evidence in a comprehensive and credible way.

Conclusions

The importance of evidence in eHealth has not been discussed as rigorously as have the diverse evaluation approaches and evaluation frameworks. Further research directed toward evidence-based evaluation can not only improve the quality of intervention studies but also facilitate successful long-term implementation of eHealth in general. We conclude that the development of more robust and comprehensive evaluation of eHealth studies or an improved validation of evaluation methods could ease the transferability of results among similar studies. Thus, the resources can be used for supplementary research in eHealth.

SUBMITTER: Enam A

PROVIDER: S-EPMC6286426 | biostudies-literature | 2018 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evidence-Based Evaluation of eHealth Interventions: Systematic Literature Review.

Enam Amia A Torres-Bonilla Johanna J Eriksson Henrik H

Journal of medical Internet research 20181123 11

<h4>Background</h4>Until now, the use of technology in health care was driven mostly by the assumptions about the benefits of electronic health (eHealth) rather than its evidence. It is noticeable that the magnitude of evidence of effectiveness and efficiency of eHealth is not proportionate to the number of interventions that are regularly conducted. Reliable evidence generated through comprehensive evaluation of eHealth interventions may accelerate the growth of eHealth for long-term successful ...[more]

PMID: 30470678

Similar Datasets

Project description:BACKGROUND:eHealth has an enormous potential to improve healthcare cost, effectiveness, and quality of care. However, there seems to be a gap between the foreseen benefits of research and clinical reality. OBJECTIVE:Our objective was to systematically review the factors influencing the outcome of eHealth interventions in terms of success and failure. METHODS:We searched the PubMed database for original peer-reviewed studies on implemented eHealth tools that reported on the factors for the success or failure, or both, of the intervention. We conducted the systematic review by following the patient, intervention, comparison, and outcome framework, with 2 of the authors independently reviewing the abstract and full text of the articles. We collected data using standardized forms that reflected the categorization model used in the qualitative analysis of the outcomes reported in the included articles. RESULTS:Among the 903 identified articles, a total of 221 studies complied with the inclusion criteria. The studies were heterogeneous by country, type of eHealth intervention, method of implementation, and reporting perspectives. The article frequency analysis did not show a significant discrepancy between the number of reports on failure (392/844, 46.5%) and on success (452/844, 53.6%). The qualitative analysis identified 27 categories that represented the factors for success or failure of eHealth interventions. A quantitative analysis of the results revealed the category quality of healthcare (n=55) as the most mentioned as contributing to the success of eHealth interventions, and the category costs (n=42) as the most mentioned as contributing to failure. For the category with the highest unique article frequency, workflow (n=51), we conducted a full-text review. The analysis of the 23 articles that met the inclusion criteria identified 6 barriers related to workflow: workload (n=12), role definition (n=7), undermining of face-to-face communication (n=6), workflow disruption (n=6), alignment with clinical processes (n=2), and staff turnover (n=1). CONCLUSIONS:The reviewed literature suggested that, to increase the likelihood of success of eHealth interventions, future research must ensure a positive impact in the quality of care, with particular attention given to improved diagnosis, clinical management, and patient-centered care. There is a critical need to perform in-depth studies of the workflow(s) that the intervention will support and to perceive the clinical processes involved.

Project description:BACKGROUND:Dyadic psychosocial interventions have been found beneficial both for people coping with mental or physical health conditions as well as their family members and friends who provide them with support. Delivering these interventions via electronic health (eHealth) may help increase their scalability. OBJECTIVE:This scoping review aimed to provide the first comprehensive overview of dyadic eHealth interventions for individuals of all ages affected by mental or physical illness and their family members or friends who support them. The goal was to understand how dyadic eHealth interventions have been used and to highlight areas of research needed to advance dyadic eHealth intervention development and dissemination. METHODS:A comprehensive electronic literature search of PubMed, EMBASE, Cochrane, Cumulative Index to Nursing and Allied Health Literature, and PsycINFO was conducted for articles published in the English language through March 2019. Eligible records described a psychosocial eHealth intervention that intervened with both care recipients and their support person. RESULTS:A total of 7113 records were reviewed of which 101 met eligibility criteria. There were 52 unique dyadic eHealth interventions identified, which were tested across 73 different trials. Of the unique interventions, 33 were conducted among dyads of children and their supporting parent, 1 was conducted with an adolescent-young adult care recipient population, and the remaining 18 were conducted among adult dyads. Interventions targeting pediatric dyads most commonly addressed a mental health condition (n=10); interventions targeting adult dyads most commonly addressed cancer (n=9). More than three-fourths of interventions (n=40) required some human support from research staff or clinicians. Most studies (n=64) specified one or more primary outcomes for care recipients, whereas less than one-fourth (n=22) specified primary outcomes for support persons. Where specified, primary outcomes were most commonly self-reported psychosocial or health factors for both care recipients (n=43) and support persons (n=18). Results of the dyadic eHealth intervention tended to be positive for care recipients, but evidence of effects for support persons was limited because of few studies specifying primary outcomes for supporters. Trials of dyadic eHealth interventions were most commonly randomized controlled trials (RCTs; n=44), and RCTs most commonly compared the dyadic eHealth intervention to usual care alone (n=22). CONCLUSIONS:This first comprehensive review of dyadic eHealth interventions demonstrates that there is substantial, diverse, and growing literature supporting this interventional approach. However, several significant gaps were identified. Few studies were designed to evaluate the unique effects of dyadic interventions relative to individual interventions. There was also limited assessment and reporting of outcomes for support persons, and there were no interventions meeting our eligibility criteria specifically targeting the needs of older adult dyads. Findings highlight areas of research opportunities for developing dyadic eHealth interventions for novel populations and for increasing access to dyadic care.

Project description:ObjectivesAcceptance and mindfulness-based interventions (A/MBIs) are recommended for people with mental health conditions. Although there is a growing evidence base supporting the effectiveness of different A/MBIs for mental health conditions, the economic case for these interventions has not been fully explored. The aim of this systematic review was to identify and appraise all available economic evidence of A/MBIs for the management of mental health conditions.MethodsEight electronic bibliographic databases (MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, Web of Science, NHS Economic Evaluation Database (EED), Database of Abstracts of Reviews of Effects (DARE), Health Technology Assessment (HTA) database, and EconLit) were searched for relevant economic evaluations published from each database's inception date until November 2017. Study selection, quality assessment, and data extraction were carried out according to published guidelines.ResultsTen relevant economic evaluations presented in 11 papers were identified. Seven of the included studies were full economic evaluations (i.e., costs and effects assessed), and three studies were partial economic evaluations (i.e., only costs were considered in the analysis). The A/MBIs that had been subjected to economic evaluation were acceptance and commitment therapy (ACT), dialectical behaviour therapy (DBT), mindfulness-based cognitive therapy (MBCT), and mindfulness-based stress reduction (MBSR). In terms of clinical presentations, the evaluation of cost-effectiveness of A/MBIs has been more focused on depression and emotional unstable personality disorder with three and four economic evaluations, respectively. Three out of seven full economic evaluations observed that A/MBIs were cost-effective for the management of mental health conditions. Nevertheless, the heterogeneity of included populations, interventions, and economic evaluation study types limits the extent to which firm conclusions can currently be made.ConclusionThis first substantive review of economic evaluations of A/MBIs indicates that more research is needed before firm conclusions can be reached on the cost-effectiveness of A/MBIs for mental health conditions.Practitioner pointsThe findings of the review provide information that may be relevant to mental health service commissioners and decision-makers as all economic evidence available on acceptance and mindfulness-based interventions for mental health conditions is summarized. Evidence relating to the cost-effectiveness and cost-saving potential of acceptance and mindfulness-based interventions is focused mainly on depression and emotional unstable personality disorder to date. Heterogeneity in the specific forms of acceptance and mindfulness-based interventions may limit generalizability of the findings. The number of health economic evaluations relating to acceptance and mindfulness-based interventions remains relatively small. Further research in this area is required.

Project description:BackgroundThe number of people with noncommunicable diseases is increasing. Noncommunicable diseases are the major cause of disability and premature mortality worldwide, associated with negative workplace outcomes such as sickness absence and reduced work productivity. There is a need to identify scalable interventions and their active components to relieve disease and treatment burden and facilitate work participation. eHealth interventions have shown potential in clinical and general populations to increase well-being and physical activity and could be well suited for workplace settings.ObjectiveWe aimed to provide an overview of the effectiveness of eHealth interventions at the workplace targeting employee health behaviors and map behavior change techniques (BCTs) used in these interventions.MethodsA systematic literature search was performed in PubMed, Embase, PsycINFO, Cochrane CENTRAL, and CINAHL in September 2020 and updated in September 2021. Extracted data included participant characteristics, setting, eHealth intervention type, mode of delivery, reported outcomes, effect sizes, and attrition rates. Quality and risk of bias of the included studies were assessed using the Cochrane Collaboration risk-of-bias 2 tool. BCTs were mapped in accordance with the BCT Taxonomy v1. The review was reported according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.ResultsIn total, 17 randomized controlled trials met the inclusion criteria. The measured outcomes, treatment and follow-up periods, content of eHealth interventions, and workplace contexts had high heterogeneity. Of the 17 studies, 4 (24%) reported unequivocally significant findings for all primary outcomes, with effect sizes ranging from small to large. Furthermore, 53% (9/17) of the studies reported mixed results, and 24% (4/17) reported nonsignificant results. The most frequently targeted behavior was physical activity (15/17, 88% of the studies); the least frequently targeted behavior was smoking (2/17, 12% of the studies). Attrition varied greatly across the studies (0%-37%). Risk of bias was high in 65% (11/17) of the studies, with some concerns in the remaining 35% (6/17). Interventions used various BCTs, and the most frequently used were feedback and monitoring (14/17, 82%), goals and planning (10/17, 59%), antecedents (10/17, 59%), and social support (7/17, 41%).ConclusionsThis review suggests that, although eHealth interventions may have potential, there are still unanswered questions regarding their effectiveness and what drives the mechanism behind these effects. Low methodological quality, high heterogeneity and complexity, the characteristics of the included samples, and often high attrition rates challenge the investigation of the effectiveness and the making of sound inferences about the effect sizes and significance of the results. To address this, new studies and methods are needed. A megastudy design in which different interventions are evaluated in the same population over the same period on the same outcomes may solve some of the challenges.Trial registrationPROSPERO CRD42020202777; https://www-crd-york-ac-uk/prospero/display_record.php?RecordID=202777.

Project description:BackgroundAccessibility is acknowledged as a key to inclusion in the Convention of Rights for People with Disabilities. An inaccessible design can result in exclusion from eHealth and cause disability among people who have impairments.ObjectiveThis scoping literature review aimed to investigate how eHealth services have been developed and evaluated regarding accessibility for people with impairments.MethodsIn line with Arksey and O'Malley's framework for scoping studies and using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR), we conducted a search in 4 databases (PubMed, Scopus, IEEE, and Web of Science) in October 2020 and an update of the search in June 2022. The search strategy was structured according to the PICO model as follows: Population/Problem, digital accessibility for users with impairment; Intervention, health care delivered by any digital solution; Comparison, not applicable; Outcome, use of and adherence to (1) Web Content Accessibility Guidelines (WCAG), (2) other accessibility guidelines, and (3) other means, for designing or evaluating accessibility in eHealth services. A Boolean search was conducted by combining terms related to accessibility and eHealth. All authors participated in screening abstracts according to the eligibility criteria. Each publication, containing a potentially relevant abstract, was read (full text) and assessed for eligibility by 2 authors independently and pairwise. Publications deemed eligible were read by all authors and discussed for consensus.ResultsA total of 8643 publications were identified. After abstract screening, 131 publications remained for full-text reading. Of those, 116 publications were excluded as they did not meet the eligibility criteria. Fifteen publications involving studies of 12 eHealth services were included in the study. Of the 15 publications, 2 provided a definition of accessibility, 5 provided an explanation of accessibility, and 8 did not provide any explanation. Five publications used the WCAG to evaluate accessibility when developing eHealth services. One publication used International Organization for Standardization (ISO) 29138, ISO 2941, and ISO/International Electrotechnical Commission (IEC) 30071-1 standards together with the Spanish Association for Standardization (UNE) 139803 standard. Eleven publications used other means to address accessibility, including text-level grading; literature review about accessibility; user tests, focus groups, interviews, and design workshops with target groups of patients, relatives, and health care professionals; and comparative analysis of existing technical solutions to provide information about useful requirements.ConclusionsAlthough a clear definition of accessibility can enhance operationalization and thus measurability when evaluating accessibility in eHealth services, accessibility was insufficiently defined in most of the included studies. Further, accessibility guidelines and standards were used to a very limited extent in the development and evaluation of eHealth services. Guidelines for developing complex interventions that include guidance for accessibility are motivated to ensure that accessibility will be considered systematically in eHealth services.

Project description:BackgroundConnected mental health, which refers to the use of technology for mental health care and technology-based therapeutic solutions, has become an established field of research. Biofeedback is one of the approaches used in connected mental health solutions, which is mainly based on the analysis of physiological indicators for the assessment and management of the psychological state. Biofeedback is recommended by many therapists and has been used for conditions including depression, insomnia, and anxiety. Anxiety is associated with several physiological symptoms, including muscle tension and breathing issues, which makes the inclusion of biofeedback useful for anxiety detection and management.ObjectiveThe aim of this study was to identify interventions using biofeedback as a part of their process for anxiety management and investigate their perceived effectiveness.MethodsA systematic literature review of publications presenting empirically evaluated biofeedback-based interventions for anxiety was conducted. The systematic literature review was based on publications retrieved from IEEE Digital Library, PubMed, ScienceDirect, and Scopus. A preliminary selection of papers was identified, examined, and filtered to include only relevant publications. Studies in the final selection were classified and analyzed to extract the modalities of use of biofeedback in the identified interventions, the types of physiological data that were collected and analyzed and the sensors used to collect them. Processes and outcomes of the empirical evaluations were also extracted.ResultsAfter final selection, 13 publications presenting different interventions were investigated. The interventions addressed either primarily anxiety disorders or anxiety associated with health issues such as migraine, Parkinson disease, and rheumatology. Solutions combined biofeedback with other techniques including virtual reality, music therapy, games, and relaxation practices and used different sensors including cardiovascular belts, wrist sensors, or stretch sensors to collect physiological data such as heart rate, respiration indicators, and movement information. The interventions targeted different cohorts including children, students, and patients. Overall, outcomes from the empirical evaluations yielded positive results and emphasized the effectiveness of connected mental health solutions using biofeedback for anxiety; however, certain unfavorable outcomes, such as interventions not having an effect on anxiety and patients' preferring traditional therapy, were reported in studies addressing patients with specific physical health issues.ConclusionsThe use of biofeedback in connected mental health interventions for the treatment and management of anxiety allows better screening and understanding of both psychological and physiological patient information, as well as of the association between the two. The inclusion of biofeedback could improve the outcome of interventions and boost their effectiveness; however, when used with patients suffering from certain physical health issues, suitability investigations are needed.

Project description:BackgroundAntimicrobial resistance (AMR) is accelerated by widespread and inappropriate use of antimicrobials. Many countries, including those in low- and middle- income contexts, have started implementing interventions to tackle AMR. However, for many interventions there is little or no economic evidence with respect to their cost-effectiveness. To help better understand the scale of this evidence gap, we conducted a systematic literature review to provide a comprehensive summary on the value for money of different interventions affecting AMR.MethodsA systematic literature review was conducted of economic evaluations on interventions addressing AMR. a narrative synthesis of findings was produced. Systematic searches for relevant studies were performed across relevant databases and grey literature sources such as unpublished studies, reports, and other relevant documents. All identified economic evaluation studies were included provided that they reported an economic outcome and stated that the analysed intervention aimed to affect AMR or antimicrobial use in the abstract. Studies that reported clinical endpoints alone were excluded. Selection for final inclusion and data extraction was performed by two independent reviewers. A quality assessment of the evidence used in the included studies was also conducted.Results28,597 articles were screened and 35 articles were identified that satisfied the inclusion criteria. The review attempted to answer the following questions: (1) What interventions to address AMR have been the subject of an economic evaluation? (2) In what types of setting (e.g. high-income, low-income, regions etc.) have these economic evaluations been focused? (3) Which interventions have been estimated to be cost-effective, and has this result been replicated in other settings/contexts? (4) What economic evaluation methods or techniques have been used to evaluate these interventions? (5) What kind and quality of data has been used in conducting economic evaluations for these interventions?DiscussionThe review is one of the first of its kind, and the most recent, to systematically review the literature on the cost-effectiveness of AMR interventions. This review addresses an important evidence gap in the economics of AMR and can assist AMR researchers' understanding of the state of the economic evaluation literature, and therefore inform future research. Systematic review registration PROSPERO (CRD42020190310).

Project description:BackgroundRapid advancements in eHealth and mobile health (mHealth) technologies have driven researchers to design and evaluate numerous technology-based interventions to promote smoking cessation. The evolving nature of cessation interventions emphasizes a strong need for knowledge synthesis.ObjectiveThis systematic review and meta-analysis aimed to summarize recent evidence from randomized controlled trials regarding the effectiveness of eHealth-based smoking cessation interventions in promoting abstinence and assess nonabstinence outcome indicators, such as cigarette consumption and user satisfaction, via narrative synthesis.MethodsWe searched for studies published in English between 2017 and June 30, 2022, in 4 databases: PubMed (including MEDLINE), PsycINFO, Embase, and Cochrane Library. Two independent reviewers performed study screening, data extraction, and quality assessment based on the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) framework. We pooled comparable studies based on the population, follow-up time, intervention, and control characteristics. Two researchers performed an independent meta-analysis on smoking abstinence using the Sidik-Jonkman random-effects model and log risk ratio (RR) as the effect measurement. For studies not included in the meta-analysis, the outcomes were narratively synthesized.ResultsA total of 464 studies were identified through an initial database search after removing duplicates. Following screening and full-text assessments, we deemed 39 studies (n=37,341 participants) eligible for this review. Of these, 28 studies were shortlisted for meta-analysis. According to the meta-analysis, SMS or app text messaging can significantly increase both short-term (3 months) abstinence (log RR=0.50, 95% CI 0.25-0.75; I2=0.72%) and long-term (6 months) abstinence (log RR=0.77, 95% CI 0.49-1.04; I2=8.65%), relative to minimal cessation support. The frequency of texting did not significantly influence treatment outcomes. mHealth apps may significantly increase abstinence in the short term (log RR=0.76, 95% CI 0.09-1.42; I2=88.02%) but not in the long term (log RR=0.15, 95% CI -0.18 to 0.48; I2=80.06%), in contrast to less intensive cessation support. In addition, personalized or interactive interventions showed a moderate increase in cessation for both the short term (log RR=0.62, 95% CI 0.30-0.94; I2=66.50%) and long term (log RR=0.28, 95% CI 0.04-0.53; I2=73.42%). In contrast, studies without any personalized or interactive features had no significant impact. Finally, the treatment effect was similar between trials that used biochemically verified or self-reported abstinence. Among studies reporting outcomes besides abstinence (n=20), a total of 11 studies reported significantly improved nonabstinence outcomes in cigarette consumption (3/14, 21%) or user satisfaction (8/19, 42%).ConclusionsOur review of 39 randomized controlled trials found that recent eHealth interventions might promote smoking cessation, with mHealth being the dominant approach. Despite their success, the effectiveness of such interventions may diminish with time. The design of more personalized interventions could potentially benefit future studies.Trial registrationPROSPERO CRD42022347104; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=347104.

Dataset Information

Evidence-Based Evaluation of eHealth Interventions: Systematic Literature Review.

Background

Objective

Methods

Results

Conclusions

Publications

Evidence-Based Evaluation of eHealth Interventions: Systematic Literature Review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets