Evaluation Methods Applied to Digital Health Interventions: What Is Being Used beyond Randomised Controlled Trials?-A Scoping Review.
Ontology highlight
ABSTRACT:
SUBMITTER: Hrynyschyn R
PROVIDER: S-EPMC9102232 | biostudies-literature | 2022 Apr
REPOSITORIES: biostudies-literature
Publications
Evaluation Methods Applied to Digital Health Interventions: What Is Being Used beyond Randomised Controlled Trials?-A Scoping Review.
Hrynyschyn Robert R Prediger Christina C Stock Christiane C Helmer Stefanie Maria SM
International journal of environmental research and public health 20220425 9
Despite the potential of digital health interventions (DHIs), evaluations of their effectiveness face challenges. DHIs are complex interventions and currently established evaluation methods, e.g., the randomised controlled trial (RCT), are limited in their application. This study aimed at identifying alternatives to RCTs as potentially more appropriate evaluation approaches. A scoping review was conducted to provide an overview of existing evaluation methods of DHIs beyond the RCT. Cochrane Cent ...[more]
Similar Datasets
Project description:ObjectiveMany adolescents struggle to access appropriate mental health care due to structural or psychological barriers. Although traditional barriers to participation (e.g., location, cost) are hypothetically reduced or removed in internet interventions, low retention reduces the likelihood that adolescents will receive the intervention dosage intended to produce beneficial effects. It is therefore key to determine what factors are associated with dropout in digital mental health interventions with adolescents both within and beyond the context of research studies.MethodsWe compare completion rates from two projects evaluating self-guided, online single-session mental health interventions (SSIs) for adolescents. One was a randomized controlled trial (RCT) in which participants were paid for participation. The other was a program evaluation project in which participants were not paid for participation. We additionally compare SSI completion rates across various demographic groups and across baseline hopelessness levels.ResultsThere was a statistically significant difference in SSI completion status between the RCT (84.75% full-completers) and the program evaluation (36.86% full-completers), X 2 (2, N = 2436) = 583.5, p < 0.05. There were no significant differences in the baseline hopelessness scores across completion statuses in either study. There were no significant differences in full-completion rates across demographic groups in either project.ConclusionAdolescents may be more likely to complete a brief digital mental health intervention in a paid, research-based context than in an unpaid, naturalistic context. Additionally, it is possible that the brevity of SSIs reduces demographic disparities related to retention by minimizing the time required to complete an intervention.
| S-EPMC8897204 | biostudies-literature
Project description:BackgroundDespite the prevalence of mobile health (mHealth) technologies and observations of their impacts on patients' health, there is still no consensus on how best to evaluate these tools for patient self-management of chronic conditions. Researchers currently do not have guidelines on which qualitative or quantitative factors to measure or how to gather these reliable data.ObjectiveThis study aimed to document the methods and both qualitative and quantitative measures used to assess mHealth apps and systems intended for use by patients for the self-management of chronic noncommunicable diseases.MethodsA scoping review was performed, and PubMed, MEDLINE, Google Scholar, and ProQuest Research Library were searched for literature published in English between January 1, 2015, and January 18, 2019. Search terms included combinations of the description of the intention of the intervention (eg, self-efficacy and self-management) and description of the intervention platform (eg, mobile app and sensor). Article selection was based on whether the intervention described a patient with a chronic noncommunicable disease as the primary user of a tool or system that would always be available for self-management. The extracted data included study design, health conditions, participants, intervention type (app or system), methods used, and measured qualitative and quantitative data.ResultsA total of 31 studies met the eligibility criteria. Studies were classified as either those that evaluated mHealth apps (ie, single devices; n=15) or mHealth systems (ie, more than one tool; n=17), and one study evaluated both apps and systems. App interventions mainly targeted mental health conditions (including Post-Traumatic Stress Disorder), followed by diabetes and cardiovascular and heart diseases; among the 17 studies that described mHealth systems, most involved patients diagnosed with cardiovascular and heart disease, followed by diabetes, respiratory disease, mental health conditions, cancer, and multiple illnesses. The most common evaluation method was collection of usage logs (n=21), followed by standardized questionnaires (n=18) and ad-hoc questionnaires (n=13). The most common measure was app interaction (n=19), followed by usability/feasibility (n=17) and patient-reported health data via the app (n=15).ConclusionsThis review demonstrates that health intervention studies are taking advantage of the additional resources that mHealth technologies provide. As mHealth technologies become more prevalent, the call for evidence includes the impacts on patients' self-efficacy and engagement, in addition to traditional measures. However, considering the unstructured data forms, diverse use, and various platforms of mHealth, it can be challenging to select the right methods and measures to evaluate mHealth technologies. The inclusion of app usage logs, patient-involved methods, and other approaches to determine the impact of mHealth is an important step forward in health intervention research. We hope that this overview will become a catalogue of the possible ways in which mHealth has been and can be integrated into research practice.
| S-EPMC7226051 | biostudies-literature
Project description:BackgroundBenefits of different types of family-centred care (FCC) interventions in neonatal intensive care units (NICUs) have been reported. However, a comprehensive review of existing FCC intervention studies was lacking.ObjectiveThis review aimed at synthesising the characteristics of FCC interventions, related outcomes and measurement methods in randomised controlled trials (RCTs) in NICU, and providing menus of options to favour implementation and further research.MethodsWe searched PubMed, EMBASE, Web of Science and the Cochrane Library up to 31 January 2022. Interventions were mapped according to five categories as defined by a previous Cochrane review. We described outcome types, measurement populations, measurement methods and timelines. Subgroup analyses were also performed.ResultsOut of 6583 studies identified, 146 met eligibility criteria. Overall, 52 (35.6%) RCTs tested more than one category of intervention, with a large variety of combinations, with the most frequent category of intervention being the educational (138 RCTs, 94.5%). We identified a total of 77 different intervention packages, and RCTs comparing the same interventions were lacking. The 146 RCTs reported on 425 different outcomes, classified in 13 major categories with parental mental health (61 RCTs, 41.8% of total RCTs) being the most frequent category in parents, and neurobehavioural/developmental outcomes being the most frequent category in newborns (62 RCTs, 42.5%). For several categories of outcomes almost every RCT used a different measurement method. Educational interventions targeting specifically staff, fathers, siblings and other family members were lacking or poorly described. Only one RCT measured outcomes in health workers, two in siblings and none considered other family members.ConclusionsA large variety of interventions, outcomes and measurement methods were used in FCC studies in NICU. The derived menus of options should be helpful for researchers and policy makers to identify interventions most suitable in each setting and to further standardise research methods.
| S-EPMC11664360 | biostudies-literature
Project description:BackgroundDigital health refers to the proper use of technology for improving the health and well-being of people and enhancing the care of patients through the intelligent processing of clinical and genetic data. Despite increasing interest in well-being in both health care and technology, there is no clear understanding of what constitutes well-being, which leads to uncertainty in how to create well-being through digital health. In an effort to clarify this uncertainty, Brey developed a framework to define problems in technology for well-being using the following four categories: epistemological problem, scope problem, specification problem, and aggregation problem.ObjectiveThis systematic scoping review aims to gain insights into how to define and address well-being in digital health.MethodsWe followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. Papers were identified from 6 databases and included if they addressed the design or evaluation of digital health and reported the enhancement of patient well-being as their purpose. These papers were divided into design and evaluation papers. We studied how the 4 problems in technology for well-being are considered per paper.ResultsA total of 117 studies were eligible for analysis (n=46, 39.3% design papers and n=71, 60.7% evaluation papers). For the epistemological problem, the thematic analysis resulted in various definitions of well-being, which were grouped into the following seven values: healthy body, functional me, healthy mind, happy me, social me, self-managing me, and external conditions. Design papers mostly considered well-being as healthy body and self-managing me, whereas evaluation papers considered the values of healthy mind and happy me. Users were rarely involved in defining well-being. For the scope problem, patients with chronic care needs were commonly considered as the main users. Design papers also regularly involved other users, such as caregivers and relatives. These users were often not involved in evaluation papers. For the specification problem, most design and evaluation papers focused on the provision of care support through a digital platform. Design papers used numerous design methods, whereas evaluation papers mostly considered pre-post measurements and randomized controlled trials. For the aggregation problem, value conflicts were rarely described.ConclusionsCurrent practice has found pragmatic ways of circumventing or dealing with the problems of digital health for well-being. Major differences exist between the design and evaluation of digital health, particularly regarding their conceptualization of well-being and the types of users studied. In addition, we found that current methodologies for designing and evaluating digital health can be improved. For optimal digital health for well-being, multidisciplinary collaborations that move beyond the common dichotomy of design and evaluation are needed.
| S-EPMC9016508 | biostudies-literature
Project description:BackgroundThis study examined the synthesis methods used in meta-analyses pooling data from observational studies (OSs) and randomised controlled trials (RCTs) from various medical disciplines.MethodsWe searched Medline via PubMed to identify reports of systematic reviews of interventions, including and pooling data from RCTs and OSs published in 110 high-impact factor general and specialised journals between 2015 and 2019. Screening and data extraction were performed in duplicate. To describe the synthesis methods used in the meta-analyses, we considered the first meta-analysis presented in each article.ResultsOverall, 132 reports were identified with a median number of included studies of 14 [9-26]. The median number of OSs was 6.5 [3-12] and that of RCTs was 3 [1-6]. The effect estimates recorded from OSs (i.e., adjusted or unadjusted) were not specified in 82% (n = 108) of the meta-analyses. An inverse-variance common-effect model was used in 2% (n = 3) of the meta-analyses, a random-effects model was used in 55% (n = 73), and both models were used in 40% (n = 53). A Poisson regression model was used in 1 meta-analysis, and 2 meta-analyses did not report the model they used. The mean total weight of OSs in the studied meta-analyses was 57.3% (standard deviation, ± 30.3%). Only 44 (33%) meta-analyses reported results stratified by study design. Of them, the results between OSs and RCTs had a consistent direction of effect in 70% (n = 31). Study design was explored as a potential source of heterogeneity in 79% of the meta-analyses, and confounding factors were investigated in only 10% (n = 13). Publication bias was assessed in 70% (n = 92) of the meta-analyses. Tau-square was reported in 32 meta-analyses with a median of 0.07 [0-0.30].ConclusionThe inclusion of OSs in a meta-analysis on interventions could provide useful information. However, considerations of several methodological and conceptual aspects of OSs, that are required to avoid misleading findings, were often absent or insufficiently reported in our sample.
| S-EPMC10880204 | biostudies-literature
Project description:BackgroundDigital care has become an essential component of health care. Interventions for patients with cancer need to be effective and safe, and digital health interventions must adhere to the same requirements.ObjectiveThe purpose of this study was to identify currently available digital health interventions developed and evaluated in randomized controlled trials (RCTs) targeting adult patients with cancer.MethodsA scoping review using the JBI methodology was conducted. The participants were adult patients with cancer, and the concept was digital health interventions. The context was open, and sources were limited to RCT effectiveness studies. The PubMed, CINAHL, Embase, Cochrane Library, Research Information Sharing Service, and KoreaMed databases were searched. Data were extracted and analyzed to achieve summarized results about the participants, types, functions, and outcomes of digital health interventions.ResultsA total of 231 studies were reviewed. Digital health interventions were used mostly at home (187/231, 81%), and the web-based intervention was the most frequently used intervention modality (116/231, 50.2%). Interventions consisting of multiple functional components were most frequently identified (69/231, 29.9%), followed by those with the self-manage function (67/231, 29%). Web-based interventions targeting symptoms with the self-manage and multiple functions and web-based interventions to treat cognitive function and fear of cancer recurrence consistently achieved positive outcomes. More studies supported the positive effects of web-based interventions to inform decision-making and knowledge. The effectiveness of digital health interventions targeting anxiety, depression, distress, fatigue, health-related quality of life or quality of life, pain, physical activity, and sleep was subject to their type and function. A relatively small number of digital health interventions specifically targeted older adults (6/231, 2.6%) or patients with advanced or metastatic cancer (22/231, 9.5%).ConclusionsThis scoping review summarized digital health interventions developed and evaluated in RCTs involving adult patients with cancer. Systematic reviews of the identified digital interventions are strongly recommended to integrate digital health interventions into clinical practice. The identified gaps in digital health interventions for cancer care need to be reflected in future digital health research.
| S-EPMC9862347 | biostudies-literature
Project description:To provide an overview and a critical appraisal of the use of responder analyses in published randomised controlled trials (RCTs) of interventions for chronic low back pain (LBP). The methodology used for the analyses, including the justification, as well as the implications of responder analyses on the conclusions was explored.A convenience sample of four systematic reviews evaluating 162 RCTs of interventions for chronic LBP was used to identify individual trials. Randomised trials were screened by two reviewers and included if they performed and reported a responder analysis (i.e. the proportion of participants achieving a pre-defined level of improvement). The cutoff value for responders, the period of follow-up, and the outcome measure used were extracted. Information on how RCT authors justified the methodology of their responder analyses was also appraised.Twenty-eight articles (17%) using 20 different definitions of responders were included in this appraisal. Justification for the definition of responders was absent in 80% of the articles. Pain was the most frequently used domain for the definition of response (50%), followed by back-specific function (30%) and a combination of pain and function (20%). A reduction in pain intensity ?50% was the most common threshold used to define responders (IQR 33-60%).Few RCTs of interventions for chronic LBP report responder analyses. Where responder analyses are used, the methods are inconsistent. When performing responder analyses authors are encouraged to follow the recommended guidelines, using empirically derived cutoffs, and present results alongside mean differences.
| S-EPMC3960418 | biostudies-other
Project description:BackgroundHome exercise regimes are a well-utilised rehabilitation intervention for many conditions; however, adherence to prescribed programmes remains low. Digital interventions are recommended as an adjunct to face-to-face interventions by the National Health Service in the UK and may offer increased exercise adherence, however the evidence for this is conflicting.MethodA systematic review was undertaken using MEDLINE and CINAHL databases using the PRISMA guidelines. Randomised controlled trials in any clinical population evaluating the adherence to prescribed home exercise interventions with and without additional digital interventions were included. Publication quality was assessed using the Cochrane Risk of Bias tool.ResultsThe search strategy returned a total of 1336 articles, of which 10 randomised controlled trials containing data for 1117 participants were eligible for inclusion. 565 participants were randomised to receive the interventions, and 552 to the control. Seven of the ten trials reported a significant difference in adherence between the control and intervention groups favouring an additional digital intervention. Three trials reported equivalent findings. These three reported longer-term outcomes, suggesting an interaction between adherence and duration of intervention. There was substantial heterogeneity in outcome assessment metrics used across the trials prohibiting formal meta-analysis. This included studies were of low to moderate quality in terms of risk of bias.ConclusionThe addition of a digital interventions to prescribed home exercise programmes can likely increase exercise adherence in the short term, with longer term effects less certain.
| S-EPMC9527092 | biostudies-literature
Project description:Good blood glucose control is important to reduce the risk of adverse effects on mothers and their offspring in women with gestational diabetes (GDM). This review examined the impact of using digital health interventions on reported glycaemic control among pregnant women with GDM and its impact on maternal and foetal outcomes. Seven databases were searched from database inception to October 31st, 2021 for randomised controlled trials that examined digital health interventions to provide services remotely for women with GDM. Two authors independently screened and assessed the studies for eligibility for inclusion. Risk of bias was independently assessed using the Cochrane Collaboration's tool. Studies were pooled using random effects model and presented as risk ratio or mean difference with 95% confidence intervals. Quality of evidence was assessed using GRADE framework. Twenty-eight randomised controlled trials that examined digital health interventions in 3,228 pregnant women with GDM were included. Moderate certainty of evidence showed that digital health interventions improved glycaemic control among pregnant women, with lower fasting plasma glucose (mean difference -0.33 mmol/L; 95% CI: -0.59 to -0.07), 2-hour post-prandial glucose (-0.49 mmol/L; -0.83 to -0.15) and HbA1c (-0.36%; -0.65 to -0.07). Among those randomised to digital health interventions, there was a lower need for caesarean delivery (Relative risk: 0.81; 0.69 to 0.95; high certainty) and foetal macrosomia (0.67; 0.48 to 0.95; high certainty). Other maternal and foetal outcomes were not significantly different between both groups. Moderate to high certainty evidence support the use of digital health interventions, as these appear to improve glycaemic control and reduce the need for caesarean delivery. However, more robust evidence is needed before it can be offered as a choice to supplement or replace clinic follow up. Systematic review registration: PROSPERO: CRD42016043009.
| S-EPMC9931335 | biostudies-literature
Project description:Food fortification (i.e., industrial fortification and biofortification) increases the micronutrient content of foods to improve population nutrition. Implementing effective fortification programs requires the generation and use of data to inform decision making. The use of digital tools and technologies (DTTs) for such purposes in broader nutrition programs is growing; however, there is limited consolidation of those used in fortification. This scoping review aimed to identify and describe DTTs used in fortification programs. We searched peer-reviewed and gray literature and conducted 17 stakeholder surveys. We then mapped DTTs identified against the fortification and nutrition data value chains. Of 11,741 articles identified, 158 met the inclusion criteria. From the included articles and stakeholder surveys, 125 DTTs were identified across three categories: software and tooling (n = 58), data and information lifecycle (n = 50), and hardware and infrastructure (n = 17). Gaps were identified in processing, post-harvest storage, aggregation, and transport nodes of the fortification value chain, and data prioritization, translation/dissemination, and decision-making nodes of the nutrition data value chain. DTTs have the potential to address challenges faced by fortification stakeholders to generate and use data to improve program decision making and nutritional impact. Further work is needed to standardize terminology, identify relevant DTTs from other sectors, and explore stakeholder needs.
| S-EPMC11829327 | biostudies-literature