Dataset Information

Applying Website Rankings to Digital Health Centers in the United States to Assess Public Engagement: Website Usability Study.

ABSTRACT:

Background

As the public increasingly uses the internet to search for resources and information regarding health and medicine, it is important that health care organizations provide adequate web resources. Website usability refers to the ease of user experience on a website. In this study, we conducted usability analyses on digital health center websites.

Objective

The primary aims of this study were to (1) replicate a preexisting usability scoring methodology for digital health centers; (2) apply and test this replicated usability scoring methodology on a sample set of digital health center websites; and (3) derive recommendations from the results on potential areas of improvements for our sample of digital health center websites.

Methods

Website usability testing was conducted from March 1, 2020, to March 15, 2020. We replicated a methodology and scoring system from previous literature and applied them to digital health center websites. Our sample included 67 digital health centers that were affiliated with US universities or hospital systems. Usability was split into the following four broad categories: accessibility, marketing, content quality, and technology. Usability tools were used to score websites in each of the four categories. The composite of the key factors of each category was used to generate a general usability and overall usability score for each website.

Results

The category with the highest average score (6.3) was content quality. The content quality score also had the highest SD (2.18) and an SE of 0.27. The lowest performing category was technology, which had an average score of 0.9. The technology score also had the smallest SD (0.07) and an SE of 0.01.

Conclusions

Our data suggest that content quality, on average, was the highest scoring variable among digital health center websites. As content is crucial to digital health knowledge, it is justified that digital health centers invest more resources into creating quality content. The overall lowest scoring variable was technology. Potential reasons for this finding include designated funding for servers, a lack of regulatory frameworks for social media presence and liability, and infrequent website audits. An easy approach for improving this variable is increasing website speed. Accessibility is another area that organizations can potentially improve. We recommend that these organizations perform periodic audits of their web presence with usability tools.

SUBMITTER: Calvano JD

PROVIDER: S-EPMC8088849 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Importance:The Centers for Medicare & Medicaid Services (CMS) Five-Star measure for nursing homes is designed with input from expert panels about the importance of multiple quality indicators. Consumers may assign different values to these indicators, creating different 5-star ratings. Objective:To compare nursing homes' rankings based on the CMS Five-Star measure with rankings based on consumers' judgment about the importance of the same quality indicators. Design, Setting, and Participants:In this quality improvement study, CMS Five-Star data were linked with a measure calculated from CMS quality indicators and consumer values obtained from a national survey. Data covered the last quarter of 2016 and the first three quarters of 2017. The study included 10?676 nursing homes, comprising 69.8% of those with reported Five-Star measures. The national survey included adults, either nursing home residents or their family members who reported being familiar with the quality of care their relative received. Data analysis was performed from January 2019 to February 2020. Main Outcomes and Measures:The contingent valuation method was administered via the survey to obtain consumers' relative values of the quality indicators, and statistical analyses were used to create the contingent valuation measure. Agreement in nursing home rankings was assessed using the Five-Star measure, which is based on weights developed by expert panels, compared with rankings based on the contingent valuation measure. Results:Among 10?676 study nursing homes with a mean (SD) of 119.4 (59.4) beds, 7845 (73.5%) were for profit, 6424 (61.8%) were part of a chain, and 8009 (75.0%) were urban. The 4310 survey respondents (mean [SD] age, 39.9 [15.6] years; 1143 [26.5%] men; 3448 [80%] white) included mostly family members (3879 participants [90.0%]). The Pearson correlation coefficient (0.65) and weighted ? statistics (0.48) indicated only moderate agreement between ranking of nursing homes' performance by the 2 measures and disagreement on ranking for approximately one-half of the nursing homes. Conclusions and Relevance:Current nursing home report cards might not reflect consumers' values and the relative importance consumers place on each of the quality indicators that compose the overall Five-Star measure. Quality report cards might be more relevant to consumers by augmenting the Five-Star measure with a measure reflecting consumers' preferences. It is unknown whether these conclusions are generalizable to other report cards, such as Hospital and Home Health Compare, without conducting similar studies for these report cards.

Project description:BackgroundDigital Mental Health (DMH) tools are an effective, readily accessible, and affordable form of mental health support. However, sustained engagement with DMH is suboptimal, with limited research on DMH engagement. The Health Action Process Approach (HAPA) is an empirically supported theory of health behavior adoption and maintenance. Whether this model also explains DMH tool engagement remains unknown.ObjectiveThis study examined whether an adapted HAPA model predicted engagement with DMH via a self-guided website.MethodsVisitors to the Mental Health America (MHA) website were invited to complete a brief survey measuring HAPA constructs. This cross-sectional study tested the adapted HAPA model with data collected using voluntary response sampling from 16,078 sessions (15,619 unique IP addresses from United States residents) on the MHA website from October 2021 through February 2022. Model fit was examined via structural equation modeling in predicting two engagement outcomes: (1) choice to engage with DMH (ie, spending 3 or more seconds on an MHA page, excluding screening pages) and (2) level of engagement (ie, time spent on MHA pages and number of pages visited, both excluding screening pages).ResultsParticipants chose to engage with the MHA website in 94.3% (15,161/16,078) of the sessions. Perceived need (β=.66; P<.001), outcome expectancies (β=.49; P<.001), self-efficacy (β=.44; P<.001), and perceived risk (β=.17-.18; P<.001) significantly predicted intention, and intention (β=.77; P<.001) significantly predicted planning. Planning was not significantly associated with choice to engage (β=.03; P=.18). Within participants who chose to engage, the association between planning with level of engagement was statistically significant (β=.12; P<.001). Model fit indices for both engagement outcomes were poor, with the adapted HAPA model accounting for only 0.1% and 1.4% of the variance in choice to engage and level of engagement, respectively.ConclusionsOur data suggest that the HAPA model did not predict engagement with DMH via a self-guided website. More research is needed to identify appropriate theoretical frameworks and practical strategies (eg, digital design) to optimize DMH tool engagement.

Project description:ObjectiveTo evaluate the perceived usability of and user engagement with a digital platform (Thought Spot) designed to enhance mental health and wellness help-seeking among transition-aged youth (TAY; 17-29-years old).Materials and methodsSurvey responses and usage patterns were collected as part of a randomized controlled trial evaluating the efficacy of Thought Spot. Participants given Thought Spot completed an adapted Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire to measure perceived usability of the platform. User engagement patterns on Thought Spot were examined using analytics data collected throughout the study (March 2018-June 2019).ResultsA total of 131 transition-aged participants completed the USE questionnaire and logged on to Thought Spot at least once. Ease of learning scored higher than ease of use, usefulness and satisfaction. Participants identified numerous strengths and challenges related to usability, visual appeal, functionality and usefulness of the content. In terms of user engagement, most participants stopped using the platform after 3 weeks. Participants searched and were interested in a variety of resources, including mental health, counselling and social services.DiscussionParticipants reported mixed experiences while using Thought Spot and exhibited low levels of long-term user engagement. User satisfaction, the willingness to recommend Thought Spot to others, and the willingness for future use appeared to be influenced by content relevance, ease of learning, available features, and other contextual factors. Analysis of the types of resources viewed and searches conducted by TAY end-users provided insight into their behaviour and needs.ConclusionUsers had mixed perceptions about the usability of Thought Spot, which may have contributed to the high attrition rate. User satisfaction and engagement appears to be influenced by content relevance, ease of learning, and the types of features available. Further investigation to understand the contextual factors that affect TAYs' adoption and engagement with digital mental health tools is required.

Project description:IntroductionParkinson's disease (PD) is a neurodegenerative disorder which requires complex medication regimens to mitigate motor symptoms. The use of digital health technology systems (DHTSs) to collect mobility and medication data provides an opportunity to objectively quantify the effect of medication on motor performance during day-to-day activities. This insight could inform clinical decision-making, personalise care, and aid self-management. This study investigates the feasibility and usability of a multi-component DHTS to remotely assess self-reported medication adherence and monitor mobility in people with Parkinson's (PwP).MethodsThirty participants with PD [Hoehn and Yahr stage I (n = 1) and II (n = 29)] were recruited for this cross-sectional study. Participants were required to wear, and where appropriate, interact with a DHTS (smartwatch, inertial measurement unit, and smartphone) for seven consecutive days to assess medication adherence and monitor digital mobility outcomes and contextual factors. Participants reported their daily motor complications [motor fluctuations and dyskinesias (i.e., involuntary movements)] in a diary. Following the monitoring period, participants completed a questionnaire to gauge the usability of the DHTS. Feasibility was assessed through the percentage of data collected, and usability through analysis of qualitative questionnaire feedback.ResultsAdherence to each device exceeded 70% and ranged from 73 to 97%. Overall, the DHTS was well tolerated with 17/30 participants giving a score > 75% [average score for these participants = 89%, from 0 (worst) to 100 (best)] for its usability. Usability of the DHTS was significantly associated with age (ρ = -0.560, BCa 95% CI [-0.791, -0.207]). This study identified means to improve usability of the DHTS by addressing technical and design issues of the smartwatch. Feasibility, usability and acceptability were identified as key themes from PwP qualitative feedback on the DHTS.ConclusionThis study highlighted the feasibility and usability of our integrated DHTS to remotely assess medication adherence and monitor mobility in people with mild-to-moderate Parkinson's disease. Further work is necessary to determine whether this DHTS can be implemented for clinical decision-making to optimise management of PwP.

Project description:BACKGROUND:This paper presents the protocol for the National Institute of Mental Health (NIMH)-funded University of Washington's ALACRITY (Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness) Center (UWAC), which uses human-centered design (HCD) methods to improve the implementation of evidence-based psychosocial interventions (EBPIs). We propose that usability-the degree to which interventions and implementation strategies can be used with ease, efficiency, effectiveness, and satisfaction-is a fundamental, yet poorly understood determinant of implementation. OBJECTIVE:We present a novel Discover, Design/Build, and Test (DDBT) framework to study usability as an implementation determinant. DDBT will be applied across Center projects to develop scalable and efficient implementation strategies (eg, training tools), modify existing EBPIs to enhance usability, and create usable and nonburdensome decision support tools for quality delivery of EBPIs. METHODS:Stakeholder participants will be implementation practitioners/intermediaries, mental health clinicians, and patients with mental illness in nonspecialty mental health settings in underresourced communities. Three preplanned projects and 12 pilot studies will employ the DDBT model to (1) identify usability challenges in implementing EBPIs in underresourced settings; (2) iteratively design solutions to overcome these challenges; and (3) compare the solution to the original version of the EPBI or implementation strategy on usability, quality of care, and patient-reported outcomes. The final products from the center will be a streamlined modification and redesign model that will improve the usability of EBPIs and implementation strategies (eg, tools to support EBPI education and decision making); a matrix of modification targets (ie, usability issues) that are both common and unique to EBPIs, strategies, settings, and patient populations; and a compilation of redesign strategies and the relative effectiveness of the redesigned solution compared to the original EBPI or strategy. RESULTS:The UWAC received institutional review board approval for the three separate studies in March 2018 and was funded in May 2018. CONCLUSIONS:The outcomes from this center will inform the implementation of EBPIs by identifying cross-cutting features of EBPIs and implementation strategies that influence the use and acceptability of these interventions, actively involving stakeholder clinicians and implementation practitioners in the design of the EBPI modification or implementation strategy solution and identifying the impact of HCD-informed modifications and solutions on intervention effectiveness and quality. TRIAL REGISTRATION:ClinicalTrials.gov NCT03515226 (https://clinicaltrials.gov/ct2/show/NCT03515226), NCT03514394 (https://clinicaltrials.gov/ct2/show/NCT03514394), and NCT03516513 (https://clinicaltrials.gov/ct2/show/NCT03516513). INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID):DERR1-10.2196/14990.