An Iterative, Mixed Usability Approach Applied to the Telekit System from the Danish TeleCare North Trial.
Ontology highlight
ABSTRACT:
SUBMITTER: Lilholt PH
PROVIDER: S-EPMC5128708 | biostudies-literature | 2016
REPOSITORIES: biostudies-literature
Publications
An Iterative, Mixed Usability Approach Applied to the Telekit System from the Danish TeleCare North Trial.
Lilholt Pernille Heyckendorff PH Schaarup Clara C Hejlesen Ole Kristian OK
International journal of telemedicine and applications 20161116
<i>Objective</i>. The aim of the present study is to evaluate the usability of the telehealth system, coined Telekit, by using an iterative, mixed usability approach. <i>Materials and Methods</i>. Ten double experts participated in two heuristic evaluations (HE1, HE2), and 11 COPD patients attended two think-aloud tests. The double experts identified usability violations and classified them into Jakob Nielsen's heuristics. These violations were then translated into measurable values on a scale o ...[more]
Similar Datasets
Project description:Although patients express an interest in using mobile health (mHealth) interventions to manage their health and chronic conditions, many current mHealth interventions are difficult to use. Usability testing is critical for the success of novel mHealth interventions. Researchers recognize the utility of using qualitative and quantitative approaches for usability testing, but many mHealth researchers lack the awareness of integration approaches from advances in mixed methods research that can add value to mHealth technology. As efficient usability testing proceeds iteratively, we introduce a novel mixed methods design developed specifically for mHealth researchers. The iterative convergent mixed methods design involves simultaneous qualitative and quantitative data collection and analysis that continues cyclically through multiple rounds of mixed methods data collection and analysis until the mHealth technology under evaluation is found to work to the satisfaction of the researcher. In cyclical iterations, early development is more qualitatively driven but progressively becomes more quantitatively driven. Using this design, mHealth researchers can leverage mixed methods integration procedures in the research question, data collection, data analysis, interpretation, and dissemination dimensions. This study demonstrates how the iterative convergent mixed methods design provides a novel framework for generating unique insights into multifaceted phenomena impacting mHealth usability. Understanding these practices can help developers and researchers leverage the strengths of an integrated mixed methods design.
| S-EPMC6658163 | biostudies-literature
Project description:PurposeResults from the Danish cluster-randomized trial of telehealthcare to 1,225 patients with chronic obstructive pulmonary disease (COPD), the Danish Telecare North Trial, concluded that the telehealthcare solution was unlikely to be cost-effective, by applying international willingness-to-pay threshold values. The purpose of this article was to assess potential sources of variation across subgroups, which could explain overall cost-effectiveness results or be utilized in future economic studies in telehealthcare research.MethodsFirst, the cost-structures and cost-effectiveness across COPD severities were analyzed. Second, five additional subgroup analyses were conducted, focusing on differences in cost-effectiveness across a set of comorbidities, age-groups, genders, resource patterns (resource use in the social care sector prior to randomization), and delivery sites. All subgroups were investigated post hoc. In analyzing cost-effectiveness, two separate linear mixed-effects models with treatment-by-covariate interactions were applied: one for quality-adjusted life-year (QALY) gain and one for total healthcare and social sector costs. Probabilistic sensitivity analysis was used for each subgroup result in order to quantify the uncertainty around the cost-effectiveness results.ResultsThe study concludes that, across the COPD severities, patients with severe COPD (GOLD 3 classification) are likely to be the most cost-effective group. This is primarily due to lower hospital-admission and primary-care costs. Telehealthcare for patients younger than 60 years is also more likely to be cost-effective than for older COPD patients. Overall, results indicate that existing resource patterns of patients and variations in delivery-site practices might have a strong influence on cost-effectiveness, possibly stronger than the included health or sociodemographic sources of heterogeneity.ConclusionFuture research should focus more on sources of heterogeneity found in the implementation context and the way telehealthcare is adopted (eg, by integrating formative evaluation into cost-effectiveness analyses).Trial registrationClinicaltrials.gov, NCT01984840.
| S-EPMC5508816 | biostudies-literature
Project description:OBJECTIVE:This study aimed to assess the cost-effectiveness of telehealthcare in heart failure patients as add-on to usual care. DESIGN:A cost-utility analysis was conducted from a public payer perspective alongside the randomised controlled TeleCare North trial. SETTING:The North Denmark Region, Denmark. PARTICIPANTS:The study included 274 heart failure patients with self-reported New York Heart Association class II-IV. INTERVENTIONS:Patients in the intervention group were provided with a Telekit consisting of a tablet, a digital blood pressure monitor, and a scale and were instructed to perform measurements one to two times a week. The responsibility of the education, instructions and monitoring of the heart failure (HF) patients was placed on municipality nurses trained in HF and telemonitoring. Both groups received usual care. OUTCOME MEASURES:Cost-effectiveness was reported as incremental net monetary benefit (NMB). A micro-costing approach was applied to evaluate the derived savings in the first year in the public health sector. Quality-adjusted life-years (QALY) gained were estimated using the EuroQol 5-Dimensions 5-Levels questionnaire at baseline and at a 1-year follow-up. RESULTS:Data for 274 patients were included in the main analysis. The telehealthcare solution provided a positive incremental NMB of £5164. The 1-year adjusted QALY difference between the telehealthcare solution and the usual care group was 0.0034 (95% CI: -0.0711 to 0.0780). The adjusted difference in costs was -£5096 (95% CI: -8736 to -1456) corresponding to a reduction in total healthcare costs by 35%. All sensitivity analyses showed the main results were robust. CONCLUSIONS:The TeleCare North solution for monitoring HF was highly cost-effective. There were significant cost savings on hospitalisations, primary care contacts and total costs. TRIAL REGISTRATION NUMBER:ClinicalTrials.gov: NCT02860013.
| S-EPMC7045102 | biostudies-literature
Project description:OBJECTIVES:To investigate the cost-effectiveness of a telehealthcare solution in addition to usual care compared with usual care. DESIGN:A 12-month cost-utility analysis conducted alongside a cluster-randomised trial. SETTING:Community-based setting in the geographical area of North Denmark Region in Denmark. PARTICIPANTS:26 municipality districts define randomisation clusters with 13 districts in each arm. 1225 patients with chronic obstructive pulmonary disease were enrolled, of which 578 patients were randomised to telehealthcare and 647 to usual care. INTERVENTIONS:In addition to usual care, patients in the intervention group received a set of telehealthcare equipment and were monitored by a municipality-based healthcare team. Patients in the control group received usual care. MAIN OUTCOME MEASURE:Incremental costs per quality-adjusted life-years gained from baseline up to 12 months follow-up. RESULTS:From a healthcare and social sector perspective, the adjusted mean difference in total costs between telehealthcare and usual care was €728 (95% CI -754 to 2211) and the adjusted mean difference in quality-adjusted life-years gained was 0.0132 (95% CI -0.0083 to 0.0346). The incremental cost-effectiveness ratio was €55 327 per quality-adjusted life-year gained. Decision-makers should be willing to pay more than €55 000 to achieve a probability of cost-effectiveness >50%. This conclusion is robust to changes in the definition of hospital contacts and reduced intervention costs. Only in the most optimistic scenario combining the effects of all sensitivity analyses, does the incremental cost-effectiveness ratio fall below the UK thresholds values (€21 068 per quality-adjusted life-year). CONCLUSIONS:Telehealthcare is unlikely to be a cost-effective addition to usual care, if it is offered to all patients with chronic obstructive pulmonary disease and if the willingness-to-pay threshold values from the National Institute for Health and Care Excellence are applied. TRIAL REGISTRATION:Clinicaltrials.gov, NCT01984840, 14 November 2013.
| S-EPMC5541337 | biostudies-literature
Project description:ObjectiveTo assess the effect of telehealthcare compared with usual practice in patients with chronic obstructive pulmonary disease (COPD).DesignA cluster-randomised trial with 26 municipal districts that were randomly assigned either to an intervention group whose members received telehealthcare in addition to usual practice or to a control group whose members received usual practice only (13 districts in each arm).SettingTwenty-six municipal districts in the North Denmark Region of Denmark.ParticipantsPatients who fulfilled the Global Initiative for COPD guidelines and one of the following criteria: COPD Assessment Test score ?10; or Medical Research Dyspnoea Council Scale ?3;?or Modified Medical Research Dyspnoea Council Scale ?2; or ?2 exacerbations during the past 12 months.Main outcome measuresHealth-related quality of life (HRQoL) assessed by the physical component summary (PCS) and mental component summary (MCS) scores of the Short Form 36-Item Health Survey, Version 2. Data were collected at baseline and at 12?month follow-up and analysed according to the intention-to-treat principle with complete cases, n=574 (258 interventions; 316 controls) and imputed data, n=1225 (578 interventions, 647 controls) using multilevel modelling.ResultsIn the intention-to-treat analysis (n=1225), the raw mean difference in PCS from baseline to 12?month follow-up was -2.6 (SD 12.4) in the telehealthcare group and -2.8 (SD 11.9) in the usual practice group. The raw mean difference in MCS scores in the same period was -4.7 (SD 16.5) and -5.3 (SD 15.5) for telehealthcare and usual practice, respectively. The adjusted mean difference in PCS and MCS between groups at 12 months was 0.1 (95% CI -1.4 to 1.7) and 0.4 (95% CI -1.7 to 2.4), respectively.ConclusionsThe overall sample and all subgroups demonstrated no statistically significant differences in HRQoL between telehealthcare and usual practice.Trial registration numberNCT01984840; Results.
| S-EPMC5623392 | biostudies-literature
Project description:BackgroundMusculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials-the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials.ObjectiveThis study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions.MethodsWe will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles.ResultsThis study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021.ConclusionsThis study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps.International registered report identifier (irrid)DERR1-10.2196/27205.
| S-EPMC8170557 | biostudies-literature
Project description:BackgroundMusculoskeletal disorders negatively affect millions of patients worldwide, placing significant demand on health care systems. Digital technologies that improve clinical outcomes and efficiency across the care pathway are development priorities. We developed the musculoskeletal Digital Assessment Routing Tool (DART) to enable self-assessment and immediate direction to the right care.ObjectiveWe aimed to assess and resolve all serious DART usability issues to create a positive user experience and enhance system adoption before conducting randomized controlled trials for the integration of DART into musculoskeletal management pathways.MethodsAn iterative, convergent mixed methods design was used, with 22 adult participants assessing 50 different clinical presentations over 5 testing rounds across 4 DART iterations. Participants were recruited using purposive sampling, with quotas for age, habitual internet use, and English-language ability. Quantitative data collection was defined by the constructs within the International Organization for Standardization 9241-210-2019 standard, with user satisfaction measured by the System Usability Scale. Study end points were resolution of all grade 1 and 2 usability problems and a mean System Usability Scale score of ≥80 across a minimum of 3 user group sessions.ResultsAll participants (mean age 48.6, SD 15.2; range 20-77 years) completed the study. Every assessment resulted in a recommendation with no DART system errors and a mean completion time of 5.2 (SD 4.44, range 1-18) minutes. Usability problems were reduced from 12 to 0, with trust and intention to act improving during the study. The relationship between eHealth literacy and age, as explored with a scatter plot and calculation of the Pearson correlation coefficient, was performed for all participants (r=-0.2; 20/22, 91%) and repeated with a potential outlier removed (r=-0.23), with no meaningful relationships observed or found for either. The mean satisfaction for daily internet users was highest (19/22, 86%; mean 86.5, SD 4.48; 90% confidence level [CL] 1.78 or -1.78), with nonnative English speakers (6/22, 27%; mean 78.1, SD 4.60; 90% CL 3.79 or -3.79) and infrequent internet users scoring the lowest (3/22, 14%; mean 70.8, SD 5.44; 90% CL 9.17 or -9.17), although the CIs overlap. The mean score across all groups was 84.3 (SD 4.67), corresponding to an excellent system, with qualitative data from all participants confirming that DART was simple to use.ConclusionsAll serious DART usability issues were resolved, and a good level of satisfaction, trust, and willingness to act on the DART recommendation was demonstrated, thus allowing progression to randomized controlled trials that assess safety and effectiveness against usual care comparators. The iterative, convergent mixed methods design proved highly effective in fully evaluating DART from a user perspective and could provide a blueprint for other researchers of mobile health systems.International registered report identifier (irrid)RR2-10.2196/27205.
| S-EPMC9472040 | biostudies-literature
Project description:BackgroundCholera poses a significant global health burden. In Bangladesh, cholera is endemic and causes more than 100,000 cases each year. Established environmental reservoirs leave millions at risk of infection through the consumption of contaminated water. The Global Task Force for Cholera Control has called for increased environmental surveillance to detect contaminated water sources prior to human infection in an effort to reduce cases and deaths. The OmniVis rapid cholera detection device uses loop-mediated isothermal amplification and particle diffusometry detection methods integrated into a handheld hardware device that attaches to an iPhone 6 to identify and map contaminated water sources.ObjectiveThe aim of this study was to evaluate the usability of the OmniVis device with targeted end users to advance the iterative prototyping process and ultimately design a device that easily integrates into users' workflow.MethodsWater quality workers were trained to use the device and subsequently completed an independent device trial and usability questionnaire. Pretraining and posttraining knowledge assessments were administered to ensure training quality did not confound trial and questionnaire.ResultsDevice trials identified common user errors and device malfunctions including incorrect test kit insertion and device powering issues. We did not observe meaningful differences in user errors or device malfunctions accumulated per participant across demographic groups. Over 25 trials, the mean time to complete a test was 47 minutes, a significant reduction compared with laboratory protocols, which take approximately 3 days. Overall, participants found the device easy to use and expressed confidence and comfort in using the device independently.ConclusionsThese results are used to advance the iterative prototyping process of the OmniVis rapid cholera detection device so it can achieve user uptake, workflow integration, and scale to ultimately impact cholera control and elimination strategies. We hope this methodology will promote robust usability evaluations of rapid pathogen detection technologies in device development.
| S-EPMC8156127 | biostudies-literature
Project description:BackgroundMedical device development is an area facing multiple challenges, resulting in a high number of products not reaching the clinical setting. Neonatal hyperbilirubinemia, manifesting as neonatal jaundice (NNJ), is an important cause of newborn morbidity and mortality. It is important to identify infants with neonatal hyperbilirubinemia at an early stage, but currently there is a lack of tools that are both accurate and affordable.ObjectiveThis study aimed to develop a novel system to assess the presence of NNJ. The device should provide accurate results, be approved as a medical device, be easy to use, and be produced at a price that is affordable even in low-resource settings.MethodsWe used an iterative approach to develop a smartphone-based system to detect the presence of NNJ. We performed technical development, followed by clinical and usability testing in parallel, after which we initiated the regulatory processes for certification. We updated the system in each iteration, and the final version underwent a clinical validation study on healthy term newborns aged 1 to 15 days before all documentation was submitted for conformity assessment to obtain Conformité Européenne (CE) certification. We developed a system that incorporates a smartphone app, a color calibration card, and a server.ResultsThree iterations of the smartphone-based system were developed; the final version was approved as a medical device after complying with Medical Device Regulation guidelines. A total of 201 infants were included in the validation study. Bilirubin values using the system highly correlated with total serum or plasma bilirubin levels (r=0.84). The system had a high sensitivity (94%) to detect severe jaundice, defined as total serum or plasma bilirubin >250 µmol/L, and maintained a high specificity (71%).ConclusionsOur smartphone-based system has a high potential as a tool for identifying NNJ. An iterative approach to product development, conducted by working on different tasks in parallel, resulted in a functional and successful product. By adhering to the requirements for regulatory approval from the beginning of the project, we were able to develop a market-ready mobile health solution.
| S-EPMC10015352 | biostudies-literature
Project description:BackgroundReporting of medication errors is one of the essential mechanisms to identify risky health care systems and practices that lead to medication errors. Unreported medication errors are a real issue; one of the identified causes is a burdensome medication error reporting system. An anonymous and user-friendly mobile app for reporting medication errors could be an alternative method of reporting medication error in busy health care settings.ObjectiveThe objective of this paper is to report usability testing of the Medication Error Reporting App (MERA), a mobile app for reporting medication errors anonymously.MethodsQuantitative and qualitative methods were employed involving 45 different testers (pharmacists, doctors, and nurses) from a large tertiary hospital in Malaysia. Quantitative data was retrieved using task performance and rating of MERA and qualitative data were retrieved through focus group discussions. Three sessions, with 15 testers each session, were conducted from January to March 2018.ResultsThe majority of testers were pharmacists (23/45, 51%), female (35/45, 78%), and the mean age was 36 (SD 9) years. A total of 135 complete reports were successfully submitted by the testers (three reports per tester) and 79.2% (107/135) of the reports were correct. There was significant improvement in mean System Usability Scale scores in each session of the development process (P<.001) and mean time to report medication errors using the app was not significantly different between each session (P=.70) with an overall mean time of 6.7 (SD 2.4) minutes. Testers found the app easy to use, but doctors and nurses were unfamiliar with terms used especially medication process at which error occurred and type of error. Although, testers agreed the app can be used in the future for reporting, they were apprehensive about security, validation, and abuse of feedback featured in the app.ConclusionsMERA can be used to report medication errors easily by various health care personnel and it has the capacity to provide feedback on reporting. However, education on medication error reporting should be provided to doctors and nurses in Malaysia and the security of the app needs to be established to boost reporting by this method.
| S-EPMC6320434 | biostudies-literature