Dataset Information

Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis.

ABSTRACT:

Background

Meaningful variations in physician performance are not always discernible from the medical record.

Objective

We used unannounced standardized patients to measure and provide feedback on care quality and fidelity of documentation, and examined downstream effects on reimbursement claims.

Design

Static group pre-post comparison study conducted between 2017 and 2019.

Setting

Fourteen New Jersey primary care practice groups (22 practices) enrolled in Horizon BCBS's value-based program received the intervention. For claims analyses, we identified 14 additional comparison practice groups matched on county, practice size, and claims activity.

Participants

Fifty-nine of 64 providers volunteered to participate.

Intervention

Unannounced standardized patients (USPs) made 217 visits portraying patients with 1-2 focal conditions (diabetes, depression, back pain, smoking, or preventive cancer screening). After two baseline visits to a provider, we delivered feedback and conducted two follow-up visits.

Measurements

USP-completed checklists of guideline-based provider care behaviors, visit audio recordings, and provider notes were used to measure behaviors performed and documentation errors pre- and post-feedback. We also compared changes in 3-month office-based claims by actual patients between the intervention and comparison practice groups before and after feedback.

Results

Expected clinical behaviors increased from 46% to 56% (OR?=?1.53, 95% CI 1.29-1.83, p?LimitationsSelf-selection of participating practices and lack of access to prescription claims.

Conclusion

Direct observation of care identifies hidden deficits in practice and documentation, and with feedback can improve both, with concomitant effects on costs.

SUBMITTER: Schwartz A

PROVIDER: S-EPMC7859004 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis.

Schwartz Alan A Peskin Steven S Spiro Alan A Weiner Saul J SJ

Journal of general internal medicine 20200707 1

<h4>Background</h4>Meaningful variations in physician performance are not always discernible from the medical record.<h4>Objective</h4>We used unannounced standardized patients to measure and provide feedback on care quality and fidelity of documentation, and examined downstream effects on reimbursement claims.<h4>Design</h4>Static group pre-post comparison study conducted between 2017 and 2019.<h4>Setting</h4>Fourteen New Jersey primary care practice groups (22 practices) enrolled in Horizon BC ...[more]

PMID: 32638322

Similar Datasets

Project description:BackgroundThe unannounced standardized patient (USP) is the gold standard for primary health care (PHC) quality assessment but has many restrictions associated with high human and resource costs. Virtual patient (VP) is a valid, low-cost software option for simulating clinical scenarios and is widely used in medical education. It is unclear whether VP can be used to assess the quality of PHC.ObjectiveThis study aimed to examine the agreement between VP and USP assessments of PHC quality and to identify factors influencing the VP-USP agreement.MethodsEleven matched VP and USP case designs were developed based on clinical guidelines and were implemented in a convenience sample of urban PHC facilities in the capital cities of the 7 study provinces. A total of 720 USP visits were conducted, during which on-duty PHC providers who met the inclusion criteria were randomly selected by the USPs. The same providers underwent a VP assessment using the same case condition at least a week later. The VP-USP agreement was measured by the concordance correlation coefficient (CCC) for continuity scores and the weighted κ for diagnoses. Multiple linear regression was used to identify factors influencing the VP-USP agreement.ResultsOnly 146 VP scores were matched with the corresponding USP scores. The CCC for medical history was 0.37 (95% CI 0.24-0.49); for physical examination, 0.27 (95% CI 0.12-0.42); for laboratory and imaging tests, -0.03 (95% CI -0.20 to 0.14); and for treatment, 0.22 (95% CI 0.07-0.37). The weighted κ for diagnosis was 0.32 (95% CI 0.13-0.52). The multiple linear regression model indicated that the VP tests were significantly influenced by the different case conditions and the city where the test took place.ConclusionsThere was low agreement between VPs and USPs in PHC quality assessment. This may reflect the "know-do" gap. VP test results were also influenced by different case conditions, interactive design, and usability. Modifications to VPs and the reasons for the low VP-USP agreement require further study.

Project description:BackgroundThe Department of Veterans Affairs (VA) has led the industry in measuring facility performance as a critical element in improving quality of care, investing substantial resources to develop and maintain valid and cost-effective measures. The External Peer Review Program (EPRP) of the VA is the official data source for monitoring facility performance, used to prioritize the quality areas needing most attention. Facility performance measurement has significantly improved preventive and chronic care, as well as overall quality; however, much variability still exists in levels of performance across measures and facilities. Audit and feedback (A&F), an important component of effective performance measurement, can help reduce this variability and improve overall performance. Previous research suggests that VA Medical Centers (VAMCs) with high EPRP performance scores tend to use EPRP data as a feedback source. However, the manner in which EPRP data are used as a feedback source by individual providers as well as service line, facility, and network leadership is not well understood. An in-depth understanding of mental models, strategies, and specific feedback process characteristics adopted by high-performing facilities is thus urgently needed.This research compares how leaders of high, low, and moderately performing VAMCs use clinical performance data from the EPRP as a feedback tool to maintain and improve quality of care.MethodsWe will conduct a qualitative, grounded theory analysis of up to 64 interviews using a novel method of sampling primary care, facility, and Veterans Integrated Service Network (VISN) leadership at high-, moderate-, and low-performing facilities. We will analyze interviews for evidence of cross-facility differences in perceptions of performance data usefulness and strategies for disseminating performance data evaluating performance, with particular attention to timeliness, individualization, and punitiveness of feedback delivery.DiscussionMost research examining feedback to improve provider and facility performance lacks a detailed understanding of the elements of effective feedback. This research will highlight the elements most commonly used at high-performing facilities and identify additional features of their successful feedback strategies not previously identified. Armed with this information, practices can implement more effective A&F interventions to improve quality of care.

Project description:BackgroundAudit and feedback aims to improve patient care by comparing healthcare performance against explicit standards. It is used to monitor and improve patient care, including through National Clinical Audit (NCA) programmes in the UK. Variability in effectiveness of audit and feedback is attributed to intervention design; separate randomised trials to address multiple questions about how to optimise effectiveness would be inefficient. We evaluated different feedback modifications to identify leading candidates for further "real-world" evaluation.MethodsUsing an online fractional factorial screening experiment, we randomised recipients of feedback from five UK NCAs to different combinations of six feedback modifications applied within an audit report excerpt: use effective comparators, provide multimodal feedback, recommend specific actions, provide optional detail, incorporate the patient voice, and minimise cognitive load. Outcomes, assessed immediately after exposure to the online modifications, included intention to enact audit standards (primary outcome, ranked on a scale of -3 to +3, tailored to the NCA), comprehension, user experience, and engagement.ResultsWe randomised 1241 participants (clinicians, managers, and audit staff) between April and October 2019. Inappropriate repeated participant completion occurred; we conservatively excluded participant entries during the relevant period, leaving a primary analysis population of 638 (51.4%) participants. None of the six feedback modifications had an independent effect on intention across the five NCAs. We observed both synergistic and antagonistic effects across outcomes when modifications were combined; the specific NCA and whether recipients had a clinical role had dominant influences on outcome, and there was an antagonistic interaction between multimodal feedback and optional detail. Among clinical participants, predicted intention ranged from 1.22 (95% confidence interval 0.72, 1.72) for the least effective combination in which multimodal feedback, optional detail, and reduced cognitive load were applied within the audit report, up to 2.40 (95% CI 1.88, 2.93) for the most effective combination including multimodal feedback, specific actions, patient voice, and reduced cognitive load.ConclusionPotentially important synergistic and antagonistic effects were identified across combinations of feedback modifications, audit programmes, and recipients, suggesting that feedback designers must explicitly consider how different features of feedback may interact to achieve (or undermine) the desired effects.Trial registrationInternational Standard Randomised Controlled Trial Number: ISRCTN41584028.

Project description:BackgroundMany primary care sites have implemented models to improve detection, diagnosis, and management of dementia, as per Canadian guidelines. The aim of this study is to describe the responses of clinicians, managers, and staff of sites that have implemented these models when presented with audit results, their insights on the factors that explain their results, their proposed solutions for improvement and how these align to one another.MethodsOne audit and feedback cycle was carried out in eight purposefully sampled sites in Ontario, Canada, that had previously implemented dementia care models. Audit consisted of a) chart review to assess quality of dementia care indicators, b) questionnaire to assess the physicians' knowledge, attitudes and practice toward dementia care, and c) semi-structured interviews to understand barriers and facilitators to implementing these models. Feedback was given to clinicians, managers, and staff in the form of graphic and oral presentations, followed by eight focus groups (one per site). Discussions revolved around: what audit results elicited more discussion from the participants, 2) their insights on the factors that explain their audit results, and 3) solutions they propose to improve dementia care. Deductive content and inductive thematic analyses, grounded in causal pathways models' theory was performed.FindingsThe audit and feedback process allowed the 63 participants to discuss many audit results and share their insights on a) organizational factors (lack of human resources, the importance of organized links with community services, clear roles and support from external memory clinics) and b) clinician factors (perceived competency practice and attitudes on dementia care), that could explain their audit results. Participants also provided solutions to improve dementia care in primary care (financial incentives, having clear pathways, adding tools to improve chart documentation, establish training on dementia care, and the possibility of benchmarking with other institutions). Proposed solutions were well aligned with their insights and further nuanced according to contextual details.ConclusionsThis study provides valuable information on solutions proposed by primary care clinicians, managers, and staff to improve dementia care in primary care. The solutions are grounded in clinical experience and will inform ongoing and future dementia strategies.

Dataset Information

Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis.

Background

Objective

Design

Setting

Participants

Intervention

Measurements

Results

Conclusion

Publications

Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets