Dataset Information

The High-Value Care Rounding Tool: Development and Validity Evidence.

ABSTRACT: Little is known about current practices in high-value care (HVC) bedside teaching. A lack of instruments for measuring bedside HVC behaviors confounds efforts to assess the impact of curricular interventions. The authors aimed to define observable HVC concepts by developing an instrument to measure the content and frequency of HVC discussions.The authors developed the HVC Rounding Tool in four iterative phases, using Messick's validity framework. Phases 1 and 2 were designed to collect evidence of content validity, Phases 3 and 4 to collect evidence of response process and internal structure. Phase 1 identified HVC topics within the literature. Phase 2 used a modified Delphi approach for construct definition and tool development. Through two rounds, the Delphi panel narrowed 16 HVC topics to 11 observable items, categorized into three domains (quality, cost, and patient values). Phase 3 involved rater training and creation of a codebook. Phase 4 involved three iterations of instrument piloting. Six trained raters, in pairs, observed bedside rounds during 148 patient encounters in 2016. Weighted kappas for each domain demonstrated improvement from the first to third iteration: Quality increased from 0.65 (95% CI 0.55-0.79) to 1.00, cost from 0.58 (95% CI 0.4-0.75) to 0.96 (95% CI 0.80-1.00), and patient values from 0.41 (95% CI 0.19-0.68) to 1.00. Percent positive agreement for all domains improved from 65.3% to 98.1%. This tool, the first with established validity evidence, addresses an important educational gap for measuring the translation of HVC from theoretical knowledge to bedside practice.

SUBMITTER: McDaniel CE

PROVIDER: S-EPMC6737943 | biostudies-literature | 2018 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The High-Value Care Rounding Tool: Development and Validity Evidence.

McDaniel Corrie E CE White Andrew A AA Bradford Miranda C MC Sy Carolyn D CD Chen Tiffany T Brock Doug D Foti Jeffrey J Beck Jimmy B JB

Academic medicine : journal of the Association of American Medical Colleges 20180201 2

Little is known about current practices in high-value care (HVC) bedside teaching. A lack of instruments for measuring bedside HVC behaviors confounds efforts to assess the impact of curricular interventions. The authors aimed to define observable HVC concepts by developing an instrument to measure the content and frequency of HVC discussions.The authors developed the HVC Rounding Tool in four iterative phases, using Messick's validity framework. Phases 1 and 2 were designed to collect evidence ...[more]

PMID: 28857791

Similar Datasets

Project description:BACKGROUND:Mentorship plays an essential role in enhancing the success of junior faculty. Previous evaluation tools focused on specific types of mentors or mentees. The main objective was to develop and provide validity evidence for a Mentor Evaluation Tool (MET) to assess the effectiveness of one-on-one mentoring for faculty in the academic health sciences. METHODS:Evidence was collected for the validity domains of content, internal structure and relationship to other variables. The 13 item MET was tested for internal structure evidence with 185 junior faculty from Schools of Dentistry, Medicine, Nursing, and Pharmacy. Finally, the MET was studied for additional validity evidence by prospectively enrolling mentees of three different groups of faculty (faculty nominated for, or winners of, a lifetime achievement in mentoring award; faculty graduates of a mentor training program; and faculty mentors not in either of the other two groups) at the University of California San Francisco (UCSF) and asking them to rate their mentors using the MET. Mentors and mentees were clinicians, educators and/or researchers. RESULTS:The 13 MET items mapped well to the five mentoring domains and six competencies described in the literature. The standardized Cronbach's coefficient alpha was 0.96. Confirmatory factor analysis supported a single factor (CFI = 0.89, SRMR = 0.05). The three mentor groups did not differ in the single overall assessment item (P = 0.054) or mean MET score (P = 0.288), before or after adjusting for years of mentoring. The mentorship score means were relatively high for all three groups. CONCLUSIONS:The Mentor Evaluation Tool demonstrates evidence of validity for research, clinical, educational or career mentors in academic health science careers. However, MET did not distinguish individuals nominated as outstanding mentors from other mentors. MET validity evidence can be studied further with mentor-mentee pairs and to follow prospectively the rating of mentors before and after a mentorship training program.

Project description:Currently health care pathways (the combination and order of services that a patient receives to manage their injury) following a mild traumatic brain injury vary considerably. Some clinicians lack confidence in injury recognition, management and knowing when to refer. A clinical expert group developed the Brain Injury Screening Tool (BIST) to provide guidance on health care pathways based on clinical indicators of poor recovery. The tool aims to facilitate access to specialist services (if required) to improve longer term prognosis. The tool was developed using a three-step process including: 1) domain mapping; 2) item development and 3) item testing and review. An online retrospective survey of 114 adults (>16 years) who had experienced a mild brain injury in the past 10 years was used to determine the initial psychometric properties of the 15-item symptom scale of the BIST. Participants were randomised to complete the BIST and one of two existing symptom scales; the Rivermead Post-concussion Symptom Questionnaire (RPQ) or the Sports Concussion Assessment Test (SCAT-5) symptom scale to determine concurrent validity. Participant responses to the BIST symptom scale items were used to determine scale reliability using Cronbach's alpha. A principal components analysis explored the underlying factor structure. Spearman's correlation coefficients determined concurrent validity with the RPQ and SCAT-5 symptom scales. The 15 items were found to require a reading age of 6-8 years old using readability statistics. High concurrent validity was shown against the RPQ (r = 0.91) and SCAT-5 (r = 0.90). The BIST total symptom scale (α = 0.94) and the three factors identified demonstrated excellent internal consistency: physical/emotional (α = 0.90), cognitive (α = 0.92) and vestibular-ocular (α = 0.80). This study provides evidence to support the utility, internal consistency, factor structure and concurrent validity of the BIST. Further research is warranted to determine the utility of the BIST scoring criteria and responsiveness to change in patients.

Project description:IntroductionOral Anticoagulation therapy (OAC) is highly effective in the management of thromboembolic disorders. An adequate level of knowledge is important for self-management and optimizing clinical outcomes. The Anticoagulation Knowledge Tool (AKT) was developed to assess OAC knowledge and caters for both patients prescribed direct oral anticoagulants or vitamin K antagonist (VKA). However, evidence regarding its psychometric proprieties, validity and reliability are unavailable in non-English speaking settings. For this reason, the aim of this study is to provide further evidence of validity for AKT and also developing an Italian AKT version (I-AKT) supported by evidence of validity and reliability.MethodsA multiphase study was conducted which included the following: cultural and linguistic validity; i.e. content validity; construct validity; reliability assessment. The Construct validity was performed using the contrasted group approach using three groups comprised of health care providers, patients and the general public. Furthermore, Exploratory Structural Equation Modelling (ESEM) was performed to confirm the mono-dimensional structure of the items in the AKT.ResultsIn construct validity phase 334 participants were enrolled. One-way ANOVA and post hoc analysis test demonstrated significant differences between the means knowledge scores of the three groups: 30.42±3.04 vs 23.45± 4.57 vs14.32±6.07 (Statistic F = 266.83; p < .001). ESEM analysis demonstrates the I-AKT mono-dimensionally structure with an explained variance of 56.42%. The scale also showed both good internal consistency reliability (Cronbach's α = 0.896) and test-retest reliability (r = 0.855).ConclusionThis study developed and validated I-AKT with supporting evidence for validity and reliability. The study also confirms the mono-dimensional of the items in the AKT. This suggest that the instrument can be useful in non-English setting for knowledge assessment and in potentially developing patient education materials.

Project description:PURPOSE:Develop and establish the content validity of the Behavioral Assessment Screening Tool (BAST?), a self-reported measure of behavioral and emotional symptoms after traumatic brain injury. METHODS:This was an assessment development study, including two focus groups of individuals with traumatic brain injury (n?=?11) and their family members (n?=?10) and an expert panel evaluation of content validity by experts in traumatic brain injury rehabilitation (n?=?7). We developed and assessed the Content Validity Index of the BAST?. RESULTS:The BAST? initial items (n?=?77) corresponded with an established conceptual model of behavioral dysregulation after traumatic brain injury. After expert panel evaluation and focus group feedback, the final BAST? included 66 items (60 primary, 6 branching logic) rated on a three-level ordinal scale (Never, Sometimes, Always) with reference to the past two weeks, and an Environmental Context checklist including recent major life events (n?=?23) and four open-ended questions about environmental factors. The BAST? had a high Content Validity Index of 89.3%. CONCLUSION:The BAST? is a theoretically grounded, multidimensional self-reported assessment of behavioral dysregulation after traumatic brain injury, with good content validity. Future translation into mobile health modalities could improve effectiveness and efficiency of long-term symptom monitoring post-traumatic brain injury. Future work will establish and validate the factor structure, internal consistency reliabilities and other validities of the BAST. Implications for Rehabilitation Behavioral problems after traumatic brain injury is one of the strongest contributing factors to poor mood and community integration outcomes after injury. Behavior is complex and multidimensional, making it a challenge to measure and to monitor long term. The Behavioral Assessment Screening Tool (BAST) is a patient-oriented outcome assessment developed in collaboration with individuals with traumatic brain injury, their care partners, and experts in the field of traumatic brain injury rehabilitation to be relevant and accessible for adults with traumatic brain injuries. The BAST is a long-term monitoring and screening tool for community-dwelling adults with traumatic brain injuries, to improve identification and management of behavioral and emotional sequelae.

Dataset Information

The High-Value Care Rounding Tool: Development and Validity Evidence.

Publications

The High-Value Care Rounding Tool: Development and Validity Evidence.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets