Dataset Information

Comparative Usability Analysis and Parental Preferences of Three Web-Based Knowledge Translation Tools: Multimethod Study.

ABSTRACT: BACKGROUND:Connecting parents to research evidence is known to improve health decision making. However, guidance on how to develop effective knowledge translation (KT) tools that synthesize child-health evidence into a form understandable by parents is lacking. OBJECTIVE:The aim of this study was to conduct a comparative usability analysis of three Web-based KT tools to identify differences in tool effectiveness, identify which format parents prefer, and better understand what factors affect usability for parents. METHODS:We evaluated a Cochrane plain language summary (PLS), Blogshot, and a Wikipedia page on a specific child-health topic (acute otitis media). A mixed method approach was used involving a knowledge test, written usability questionnaire, and a semistructured interview. Differences in knowledge and usability questionnaire scores for each of the KT tools were analyzed using Kruskal-Wallis tests, considering a critical significance value of P=.05. Thematic analysis was used to synthesize and identify common parent preferences among the semistructured interviews. Key elements parents wanted in a KT tool were derived through author consensus using questionnaire data and parent interviews. RESULTS:In total, 16 parents (9 female) with a mean age of 39.6 (SD 11.9) years completed the study. Parents preferred the Blogshot over the PLS and Wikipedia page (P=.002) and found the Blogshot to be the most aesthetic (P=.001) and easiest to use (P=.001). Knowledge questions and usability survey data also indicated that the Blogshot was the most preferred and effective KT tool at relaying information about the topic. Four key themes were derived from thematic analysis, describing elements parents valued in KT tools. Parents wanted tools that were (1) simple, (2) quick to access and use, and (3) trustworthy, and which (4) informed how to manage the condition. Out of the three KT tools assessed, Blogshots were the most preferred tool by parents and encompassed these four key elements. CONCLUSIONS:It is important that child health evidence be available in formats accessible and understandable by parents to improve decision making, use of health care resources, and health outcomes. Further usability testing of different KT tools should be conducted involving broader populations and other conditions (eg, acute vs chronic) to generate guidelines to improve KT tools for parents.

SUBMITTER: Anzinger H

PROVIDER: S-EPMC7101501 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparative Usability Analysis and Parental Preferences of Three Web-Based Knowledge Translation Tools: Multimethod Study.

Anzinger Harrison H Elliott Sarah Alexandra SA Hartling Lisa L

Journal of medical Internet research 20200313 3

<h4>Background</h4>Connecting parents to research evidence is known to improve health decision making. However, guidance on how to develop effective knowledge translation (KT) tools that synthesize child-health evidence into a form understandable by parents is lacking.<h4>Objective</h4>The aim of this study was to conduct a comparative usability analysis of three Web-based KT tools to identify differences in tool effectiveness, identify which format parents prefer, and better understand what fac ...[more]

PMID: 32167478

Similar Datasets

Project description:BackgroundThe field of knowledge translation (KT) has been criticized for neglecting contextual and social considerations that influence health equity. Intersectionality, a concept introduced by Black feminist scholars, emphasizes how human experience is shaped by combinations of social factors (e.g., ethnicity, gender) embedded in systemic power structures. Its use has the potential to advance equity considerations in KT. Our objective was to develop and conduct usability testing of tools to support integrating intersectionality in KT through three key phases of KT: identifying the gap; assessing barriers to knowledge use; and selecting, tailoring, and implementing interventions.MethodsWe used an integrated KT approach and assembled an interdisciplinary development committee who drafted tools. We used a mixed methods approach for usability testing with KT intervention developers that included semi-structured interviews and the System Usability Scale (SUS). We calculated an average SUS score for each tool. We coded interview data using the framework method focusing on actionable feedback. The development committee used the feedback to revise tools, which were formatted by a graphic designer.ResultsNine people working in Canada joined the development committee. They drafted an intersectionality primer and one tool that included recommendations, activities, reflection prompts, and resources for each of the three implementation phases. Thirty-one KT intervention developers from three countries participated in usability testing. They suggested the tools to be shorter, contain more visualizations, and use less jargon. Average SUS scores of the draft tools ranged between 60 and 78/100. The development committee revised and shortened all tools, and added two, one-page summary documents. The final toolkit included six documents.ConclusionsWe developed and evaluated tools to help embed intersectionality considerations in KT. These tools go beyond recommending the use of intersectionality to providing practical guidance on how to do this. Future work should develop guidance for enhancing social justice in intersectionality-enhanced KT.

Project description:BackgroundTo meet the growing importance of real-word data analysis, clinical data and biosamples must be timely made available. Feasibility platforms are often the first contact point for determining the availability of such data for specific research questions. Therefore, a user-friendly interface should be provided to enable access to this information easily. The German Medical Informatics Initiative also aims to establish such a platform for its infrastructure. Although some of these platforms are actively used, their tools still have limitations. Consequently, the Medical Informatics Initiative consortium MIRACUM (Medical Informatics in Research and Care in University Medicine) committed itself to analyzing the pros and cons of existing solutions and to designing an optimized graphical feasibility user interface.ObjectiveThe aim of this study is to identify the system that is most user-friendly and thus forms the best basis for developing a harmonized tool. To achieve this goal, we carried out a comparative usability evaluation of existing tools used by researchers acting as end users.MethodsThe evaluation included three preselected search tools and was conducted as a qualitative exploratory study with a randomized design over a period of 6 weeks. The tools in question were the MIRACUM i2b2 (Informatics for Integrating Biology and the Bedside) feasibility platform, OHDSI's (Observational Health Data Sciences and Informatics) ATLAS, and the Sample Locator of the German Biobank Alliance. The evaluation was conducted in the form of a web-based usability test (usability walkthrough combined with a web-based questionnaire) with participants aged between 26 and 63 years who work as medical doctors.ResultsIn total, 17 study participants evaluated the three tools. The overall evaluation of usability, which was based on the System Usability Scale, showed that the Sample Locator, with a mean System Usability Scale score of 77.03 (SD 20.62), was significantly superior to the other two tools (Wilcoxon test; Sample Locator vs i2b2: P=.047; Sample Locator vs ATLAS: P=.001). i2b2, with a score of 59.83 (SD 25.36), performed significantly better than ATLAS, which had a score of 27.81 (SD 21.79; Wilcoxon test; i2b2 vs ATLAS: P=.005). The analysis of the material generated by the usability walkthrough method confirmed these findings. ATLAS caused the most usability problems (n=66), followed by i2b2 (n=48) and the Sample Locator (n=22). Moreover, the Sample Locator achieved the highest ratings with respect to additional questions regarding satisfaction with the tools.ConclusionsThis study provides data to develop a suitable basis for the selection of a harmonized tool for feasibility studies via concrete evaluation and a comparison of the usability of three different types of query builders. The feedback obtained from the participants during the usability test made it possible to identify user problems and positive design aspects of the individual tools and compare them qualitatively.

Project description:BACKGROUND:We explored the performance of three machine learning tools designed to facilitate title and abstract screening in systematic reviews (SRs) when used to (a) eliminate irrelevant records (automated simulation) and (b) complement the work of a single reviewer (semi-automated simulation). We evaluated user experiences for each tool. METHODS:We subjected three SRs to two retrospective screening simulations. In each tool (Abstrackr, DistillerSR, RobotAnalyst), we screened a 200-record training set and downloaded the predicted relevance of the remaining records. We calculated the proportion missed and workload and time savings compared to dual independent screening. To test user experiences, eight research staff tried each tool and completed a survey. RESULTS:Using Abstrackr, DistillerSR, and RobotAnalyst, respectively, the median (range) proportion missed was 5 (0 to 28) percent, 97 (96 to 100) percent, and 70 (23 to 100) percent for the automated simulation and 1 (0 to 2) percent, 2 (0 to 7) percent, and 2 (0 to 4) percent for the semi-automated simulation. The median (range) workload savings was 90 (82 to 93) percent, 99 (98 to 99) percent, and 85 (85 to 88) percent for the automated simulation and 40 (32 to 43) percent, 49 (48 to 49) percent, and 35 (34 to 38) percent for the semi-automated simulation. The median (range) time savings was 154 (91 to 183), 185 (95 to 201), and 157 (86 to 172) hours for the automated simulation and 61 (42 to 82), 92 (46 to 100), and 64 (37 to 71) hours for the semi-automated simulation. Abstrackr identified 33-90% of records missed by a single reviewer. RobotAnalyst performed less well and DistillerSR provided no relative advantage. User experiences depended on user friendliness, qualities of the user interface, features and functions, trustworthiness, ease and speed of obtaining predictions, and practicality of the export file(s). CONCLUSIONS:The workload savings afforded in the automated simulation came with increased risk of missing relevant records. Supplementing a single reviewer's decisions with relevance predictions (semi-automated simulation) sometimes reduced the proportion missed, but performance varied by tool and SR. Designing tools based on reviewers' self-identified preferences may improve their compatibility with present workflows. SYSTEMATIC REVIEW REGISTRATION:Not applicable.

Project description:An emerging field of knowledge translation (KT) research has begun to focus on health consumers, particularly in child health. KT tools provide health consumers with research knowledge to inform health decision-making and may foster 'effective consumers'. Thus, the purpose of this scoping review was to describe the state of the field of previously published effectiveness research on child health-related KT tools for parents/caregivers to understand the evidence-base, identify gaps, and guide future research efforts.A health research librarian developed and implemented search strategies in 8 databases. One reviewer conducted screening using pre-determined criteria. A second reviewer verified 10% of screening decisions. Data extraction was performed by one reviewer. A descriptive analysis was conducted and included patient-important outcome classification, WIDER Recommendation checklist, and methodological quality assessment.Seven thousand nine hundred fifty two independent titles and abstracts were reviewed, 2267 full-text studies were retrieved and reviewed, and 18 articles were included in the final data set. A variety of KT tools, including single- (n = 10) and multi-component tools (n = 10), were evaluated spanning acute (n = 4), chronic (n = 5) and public/population health (n = 9) child health topics. Study designs included: cross-sectional (n = 4), before-after (n = 1), controlled before-after (n = 2), cohort (n = 1), and RCTs (n = 10). The KT tools were evaluated via single primary outcome category (n = 11) and multiple primary outcome categories (n = 7). Two studies demonstrated significant positive effects on primary outcome categories; the remaining studies demonstrated mixed effects (n = 9) and no effect (n = 3). Overall, methodological quality was poor; studies lacked a priori protocols (n = 18) and sample size calculations (n = 13). Overall, intervention reporting was also poor; KT tools lacked description of theoretical underpinnings (n = 14), end-user engagement (n = 13), and preliminary research (n = 9) to inform the current effectiveness evaluation.A number of child health-related knowledge translation tools have been developed for parents/caregivers. However, numerous outcomes were used to assess impact and there is limited evidence demonstrating their effectiveness. Moreover, the methodological rigor and reporting of effectiveness studies is limited. Careful tool development involving end-users and preliminary research, including usability testing and mixed methods, prior to large-scale studies may be important to advance the science of KT for health consumers.

Project description:BackgroundGovernment sponsors of research and development, along with their funded investigators, are increasingly tasked with demonstrating evidence of knowledge use by nontraditional audiences. This requires efforts to translate their findings for effective communication. For technology-related knowledge, these audiences include clinicians, consumers, manufacturers, public policy agencies, and knowledge brokers. One potentially efficient approach is to communicate research findings through relevant national organizations. However, this requires an understanding of how such organizations view and treat research knowledge, which can be determined through knowledge-value mapping. Do knowledge values differ between national organizations representing different audiences? Can a deeper understanding of knowledge values help sponsors, investigators, and organizations better communicate research findings to stakeholders?MethodsA series of comparative case studies on knowledge-value mapping were derived through interviews with spokespersons for six national organizations. The semi-structured interviews followed a 10-item questionnaire to characterize different ways in which each organization engages with research-based knowledge. Each participating organization represents a particular stakeholder group, while all share a common interest in the research subject matter.ResultsEach national organization considers the value of the research knowledge in the context of their organization's mission and the interests of their members. All are interested in collaborating with researchers to share relevant findings, while they vary along the following dimensions of knowledge engagement: create, identify, translate, adapt, communicate, use, promote, absorptive capacity, and recommendations for facilitation.ConclusionsThe principles of knowledge translation suggest that investigators can increase use by tailoring the format and context of their findings to the absorptive capacity of nonscholars. Greater absorption should result in higher levels of knowledge awareness, interest, and use, which can then be documented. National organizations and their members, in turn, can strive to optimize their absorptive capacities regarding the state of the sciences. This combination will ensure the highest possible return on public investment in research activities. This knowledge-value mapping study concludes that national organizations are appropriate channels for communicating research findings and for meeting statutory requirements and general expectations for generating and documenting knowledge use.

Project description:BackgroundClinicians are important stakeholders in the translation of well-designed research evidence into clinical practice for optimal patient care. However, the application of knowledge translation (KT) theories and processes may present conceptual and practical challenges for clinicians. Online learning platforms are an effective means of delivering KT education, providing an interactive, time-efficient, and affordable alternative to face-to-face education programs.ObjectiveThis study investigates the availability and accessibility of online KT learning opportunities for health professionals. It also provides an analysis of the types of resources and associated disciplines retrieved by a range of KT synonyms.MethodsWe searched a range of bibliographic databases and the Internet (Google advanced option) using 9 KT terms to identify online KT learning resources. To be eligible, resources had to be free, aimed at clinicians, educational in intent, and interactive in design. Each term was searched using two different search engines. The details of the first 100 websites captured per browser (ie, n=200 results per term) were entered into EndNote. Each site was subsequently visited to determine its status as a learning resource. Eligible websites were appraised for quality using the AACODS (Authority, Accuracy, Coverage, Objectivity, Date, Significance) tool.ResultsWe identified 971 unique websites via our multiple search strategies. Of these, 43 were health-related and educational in intent. Once these sites were evaluated for interactivity, a single website matched our inclusion criteria (Dementia Knowledge Translation Learning Centre).ConclusionsKT is an important but complex system of processes. These processes overlap with knowledge, practice, and improvement processes that go by a range of different names. For clinicians to be informed and competent in KT, they require better access to free learning opportunities. These resources should be designed from the viewpoint of the clinician, presenting KT's multifaceted theories and processes in an engaging, interactive way. This learning should empower clinicians to contextualize and apply KT strategies within their own care settings.

Project description:BackgroundAcute gastroenteritis (AGE) in children is a leading cause of emergency department (ED) visits, resulting in substantial health care costs and stress for families and caregivers. The majority of pediatric AGE cases are caused by viral infections and can be managed at home using strategies to prevent dehydration. To increase knowledge of, and support health decision-making for, pediatric AGE, we developed a knowledge translation (KT) tool (fully automated web-based whiteboard animation video).ObjectiveThe aim of this study was to assess the potential effectiveness of the web-based KT tool in terms of knowledge, health care decision-making, use of resources, and perceived benefit and value.MethodsA convenience sample of parents was recruited between December 18, 2020, and August 10, 2021. Parents were recruited in the ED of a pediatric tertiary care hospital and followed for up to 14 days after the ED visit. The eligibility criteria included parent or legal guardian of a child aged <16 years presenting to the ED with an acute episode of diarrhea or vomiting, ability to communicate in English, and agreeable to follow-up via email. Parents were randomized to receive the web-based KT tool (intervention) about AGE or a sham video (control) during their ED visit. The primary outcome was knowledge assessed before the intervention (baseline), immediately after the intervention, and at follow-up 4 to 14 days after ED discharge. Other outcomes included decision regret, health care use, and KT tool usability and satisfaction. The intervention group participants were invited to participate in a semistructured interview to gather additional feedback about the KT tool.ResultsA total of 103 parents (intervention: n=51, 49.5%, and control: n=52, 50.5%) completed the baseline and postintervention assessments. Of these 103 parents, 78 (75.7%; intervention: n=36, 46%, and control: n=42, 54%) completed the follow-up questionnaire. Knowledge scores after the intervention (mean 8.5, SD 2.6 vs mean 6.3, SD 1.7; P<.001) and at follow-up (mean 9.1, SD 2.7 vs mean 6.8, SD 1.6; P<.001) were significantly higher in the intervention group. After the intervention, parents in the intervention group reported greater confidence in knowledge than those in the control group. No significant difference in decision regret was found at any time point. Parents rated the KT tool higher than the sham video across 5 items assessing usability and satisfaction.ConclusionsThe web-based KT tool improved parental knowledge about AGE and confidence in their knowledge, which are important precursors to behavior change. Further research is needed into understanding what information and delivery format as well as other factors influence parents' decision-making regarding their child's health.Trial registrationClinicalTrials.gov NCT03234777; https://clinicaltrials.gov/ct2/show/NCT03234777.International registered report identifier (irrid)RR2-10.1186/s40814-018-0318-0.

Dataset Information

Comparative Usability Analysis and Parental Preferences of Three Web-Based Knowledge Translation Tools: Multimethod Study.

Publications

Comparative Usability Analysis and Parental Preferences of Three Web-Based Knowledge Translation Tools: Multimethod Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets