Dataset Information

Applying Website Rankings to Digital Health Centers in the United States to Assess Public Engagement: Website Usability Study.

ABSTRACT:

Background

As the public increasingly uses the internet to search for resources and information regarding health and medicine, it is important that health care organizations provide adequate web resources. Website usability refers to the ease of user experience on a website. In this study, we conducted usability analyses on digital health center websites.

Objective

The primary aims of this study were to (1) replicate a preexisting usability scoring methodology for digital health centers; (2) apply and test this replicated usability scoring methodology on a sample set of digital health center websites; and (3) derive recommendations from the results on potential areas of improvements for our sample of digital health center websites.

Methods

Website usability testing was conducted from March 1, 2020, to March 15, 2020. We replicated a methodology and scoring system from previous literature and applied them to digital health center websites. Our sample included 67 digital health centers that were affiliated with US universities or hospital systems. Usability was split into the following four broad categories: accessibility, marketing, content quality, and technology. Usability tools were used to score websites in each of the four categories. The composite of the key factors of each category was used to generate a general usability and overall usability score for each website.

Results

The category with the highest average score (6.3) was content quality. The content quality score also had the highest SD (2.18) and an SE of 0.27. The lowest performing category was technology, which had an average score of 0.9. The technology score also had the smallest SD (0.07) and an SE of 0.01.

Conclusions

Our data suggest that content quality, on average, was the highest scoring variable among digital health center websites. As content is crucial to digital health knowledge, it is justified that digital health centers invest more resources into creating quality content. The overall lowest scoring variable was technology. Potential reasons for this finding include designated funding for servers, a lack of regulatory frameworks for social media presence and liability, and infrequent website audits. An easy approach for improving this variable is increasing website speed. Accessibility is another area that organizations can potentially improve. We recommend that these organizations perform periodic audits of their web presence with usability tools.

SUBMITTER: Calvano JD

PROVIDER: S-EPMC8088849 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe Association of American Medical Colleges has recently issued recommendations for the upcoming 2022-2023 application cycle that residency programs should conduct all interviews for this upcoming application cycle over the web. In light of these recommendations, many students will have limited exposure to anesthesiology programs and will rely on information gleaned digitally. This change means that the aspects of program websites used to provide information, such as size, structure, location, requirements, and contact information, will be crucial in helping prospective residents decide where and how to apply in the future. An evaluation of website usability, which includes initial appearance along with factors that influence its ease of navigation and convenience of use, can thus be applied to anesthesiology residency websites. Areas of need can be targeted to increase web presence and provide effective pathways to exhibit the different attributes of their programs to future applicants.ObjectiveThis study aimed to compile a list of US anesthesiology residency programs and their websites while objectively analyzing the websites using a formally published usability scoring system, as well as to identify positive and negative trends to offer areas of improvement among anesthesiology residency websites.MethodsWe included only 114 US anesthesiology residency program websites in our sample set, since some websites we analyzed showed errors or inconclusive. Website usability was separated into 4 distinct categories for analysis based on methodology outlined in previous literature on both health care website usability and residency website usability. The 4 categories were Accessibility, Marketing, Content Quality, and Technology. Each website was then analyzed and scored based on key components highlighted within the 4 categories. The multiple factors were then graded using a percentage system to create a comprehensive score for each program.ResultsThe highest scoring category was Content Quality (mean 4.7, SD 2.48, SE 0.23). The lowest scoring category was Technology (mean 0.9, SD 0.38, SE 0.04).ConclusionsThrough the application of a health care website usability framework, multiple anesthesiology residency programs were analyzed and scored in the areas of Accessibility, Marketing, Content Quality, and Technology, which allowed us to determine the effectiveness of the usability of these websites to convey information to their end user. Websites must communicate vital information, with usability at the forefront, to continue to grow, especially as the United States faces challenges due to the COVID-19 pandemic. Our recommendation is that anesthesiology programs should strive to improve website usability to increase the ease by which applicants can collect vital information about anesthesiology programs. A few proposed solutions include making changes such as decreasing error pages on websites, migrating away from using in-line cascading style sheets, and improving web page loading speeds to improve the Technology category.

Project description:Importance:The Centers for Medicare & Medicaid Services (CMS) Five-Star measure for nursing homes is designed with input from expert panels about the importance of multiple quality indicators. Consumers may assign different values to these indicators, creating different 5-star ratings. Objective:To compare nursing homes' rankings based on the CMS Five-Star measure with rankings based on consumers' judgment about the importance of the same quality indicators. Design, Setting, and Participants:In this quality improvement study, CMS Five-Star data were linked with a measure calculated from CMS quality indicators and consumer values obtained from a national survey. Data covered the last quarter of 2016 and the first three quarters of 2017. The study included 10?676 nursing homes, comprising 69.8% of those with reported Five-Star measures. The national survey included adults, either nursing home residents or their family members who reported being familiar with the quality of care their relative received. Data analysis was performed from January 2019 to February 2020. Main Outcomes and Measures:The contingent valuation method was administered via the survey to obtain consumers' relative values of the quality indicators, and statistical analyses were used to create the contingent valuation measure. Agreement in nursing home rankings was assessed using the Five-Star measure, which is based on weights developed by expert panels, compared with rankings based on the contingent valuation measure. Results:Among 10?676 study nursing homes with a mean (SD) of 119.4 (59.4) beds, 7845 (73.5%) were for profit, 6424 (61.8%) were part of a chain, and 8009 (75.0%) were urban. The 4310 survey respondents (mean [SD] age, 39.9 [15.6] years; 1143 [26.5%] men; 3448 [80%] white) included mostly family members (3879 participants [90.0%]). The Pearson correlation coefficient (0.65) and weighted ? statistics (0.48) indicated only moderate agreement between ranking of nursing homes' performance by the 2 measures and disagreement on ranking for approximately one-half of the nursing homes. Conclusions and Relevance:Current nursing home report cards might not reflect consumers' values and the relative importance consumers place on each of the quality indicators that compose the overall Five-Star measure. Quality report cards might be more relevant to consumers by augmenting the Five-Star measure with a measure reflecting consumers' preferences. It is unknown whether these conclusions are generalizable to other report cards, such as Hospital and Home Health Compare, without conducting similar studies for these report cards.

Project description:BackgroundMany individuals living with dementia want to live in their own homes for as long as possible. To do so, they frequently require assistance with activities of daily living, which is often provided by friends and relatives acting as informal care partners. In Canada, many informal care partners are currently overworked and overwhelmed. Although community-based dementia-inclusive resources are available to support them, care partners often struggle to find them. Dementia613.ca was created to make the process of finding community dementia-inclusive resources simpler and more straightforward by bringing them together in one eHealth website.ObjectiveThe objective of our study was to determine if dementia613.ca is meeting the goal of connecting care partners and persons living with dementia to dementia-inclusive resources in their community.MethodsA review and assessment of the website was conducted using 3 evaluation methods: web analytics, questionnaires, and task analysis. Google Analytics was used to collect data related to website use over a 9-month period. Data on site content and user characteristics were collected. Furthermore, 2 web-based self-administered questionnaires were developed: one intended for care partners and persons living with dementia, and the other intended for businesses and organizations interested in serving persons living with dementia. Both gathered data on user characteristics and included standard questions used in website evaluations. Responses were collected over a 6-month period. Scenarios, tasks, and questions were developed for the moderated, remote, and task-analysis sessions. These tasks and questions determined how effectively persons living with dementia and their care partners can use dementia613.ca. Overall, 5 sessions were held with persons experiencing moderate cognitive decline and with care partners of persons living with dementia.ResultsThis evaluation showed that the idea behind dementia613.ca is strong and appeals to persons living with dementia, their care partners, and the businesses and organizations serving this market. Participants indicated that it is a useful community resource that meets a previously unfulfilled need in the area, and highlighted the benefits of bringing community resources together on 1 website. In our questionnaire, >60% (19/29, 66%) of people living with dementia and their care partners and 70% (7/10) of businesses and organizations agreed that the website made it easier to find relevant dementia-inclusive resources. There is room for improvement; participants indicated that the navigation and search features could be further developed.ConclusionsWe believe that the dementia613.ca model could be used to inspire and guide the creation of dementia resource websites in other regions in Ontario and beyond. The framework behind it is generalizable and could be replicated to help care partners and persons living with dementia find local resources more easily.

Project description:BackgroundDigital Mental Health (DMH) tools are an effective, readily accessible, and affordable form of mental health support. However, sustained engagement with DMH is suboptimal, with limited research on DMH engagement. The Health Action Process Approach (HAPA) is an empirically supported theory of health behavior adoption and maintenance. Whether this model also explains DMH tool engagement remains unknown.ObjectiveThis study examined whether an adapted HAPA model predicted engagement with DMH via a self-guided website.MethodsVisitors to the Mental Health America (MHA) website were invited to complete a brief survey measuring HAPA constructs. This cross-sectional study tested the adapted HAPA model with data collected using voluntary response sampling from 16,078 sessions (15,619 unique IP addresses from United States residents) on the MHA website from October 2021 through February 2022. Model fit was examined via structural equation modeling in predicting two engagement outcomes: (1) choice to engage with DMH (ie, spending 3 or more seconds on an MHA page, excluding screening pages) and (2) level of engagement (ie, time spent on MHA pages and number of pages visited, both excluding screening pages).ResultsParticipants chose to engage with the MHA website in 94.3% (15,161/16,078) of the sessions. Perceived need (β=.66; P<.001), outcome expectancies (β=.49; P<.001), self-efficacy (β=.44; P<.001), and perceived risk (β=.17-.18; P<.001) significantly predicted intention, and intention (β=.77; P<.001) significantly predicted planning. Planning was not significantly associated with choice to engage (β=.03; P=.18). Within participants who chose to engage, the association between planning with level of engagement was statistically significant (β=.12; P<.001). Model fit indices for both engagement outcomes were poor, with the adapted HAPA model accounting for only 0.1% and 1.4% of the variance in choice to engage and level of engagement, respectively.ConclusionsOur data suggest that the HAPA model did not predict engagement with DMH via a self-guided website. More research is needed to identify appropriate theoretical frameworks and practical strategies (eg, digital design) to optimize DMH tool engagement.

Project description:ObjectiveTo evaluate the perceived usability of and user engagement with a digital platform (Thought Spot) designed to enhance mental health and wellness help-seeking among transition-aged youth (TAY; 17-29-years old).Materials and methodsSurvey responses and usage patterns were collected as part of a randomized controlled trial evaluating the efficacy of Thought Spot. Participants given Thought Spot completed an adapted Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire to measure perceived usability of the platform. User engagement patterns on Thought Spot were examined using analytics data collected throughout the study (March 2018-June 2019).ResultsA total of 131 transition-aged participants completed the USE questionnaire and logged on to Thought Spot at least once. Ease of learning scored higher than ease of use, usefulness and satisfaction. Participants identified numerous strengths and challenges related to usability, visual appeal, functionality and usefulness of the content. In terms of user engagement, most participants stopped using the platform after 3 weeks. Participants searched and were interested in a variety of resources, including mental health, counselling and social services.DiscussionParticipants reported mixed experiences while using Thought Spot and exhibited low levels of long-term user engagement. User satisfaction, the willingness to recommend Thought Spot to others, and the willingness for future use appeared to be influenced by content relevance, ease of learning, available features, and other contextual factors. Analysis of the types of resources viewed and searches conducted by TAY end-users provided insight into their behaviour and needs.ConclusionUsers had mixed perceptions about the usability of Thought Spot, which may have contributed to the high attrition rate. User satisfaction and engagement appears to be influenced by content relevance, ease of learning, and the types of features available. Further investigation to understand the contextual factors that affect TAYs' adoption and engagement with digital mental health tools is required.