Blending group and practice site scores to increase the reliability of physician quality information.
Ontology highlight
ABSTRACT: Objective
To use an empirical Bayesian approach, blending practice, and group quality data with physician results to increase the accuracy of quality of care measures.Data sources
Performance data on diabetes glycemic screening for 8,357 physicians collected from multiple payers as part of a statewide physician performance reporting initiative.Study design
A variance components analysis assessed the strength of group, practice, and physician effects compared with random error. We derived formulas to describe reliability and measurement error variances and calculated the optimal blend of physician, practice, and group data. We constructed a simulation to show what various methods can achieve. The value of blending strategies was assessed by simulating a common pay-for-performance criterion-performance in the top 25 percent. We estimated the proportion of physicians whose true percentage would place them in the top 20 percent but who would not receive payment based on the observed success rate.Principal findings
Blending reduced the error rate from 29.7 to 22.7 percent. Simpler empirical Bayes estimates using shrinkage alone produced no gains over simple doctor percentages.Conclusions
When good structural data about physician groups and practices exist, gains from blending can be substantial.
SUBMITTER: Rogers WH
PROVIDER: S-EPMC3922469 | biostudies-literature | 2014 Feb
REPOSITORIES: biostudies-literature
Publications
Blending group and practice site scores to increase the reliability of physician quality information.
Rogers William H WH Thornton Kristy K von Glahn Ted T
Health services research 20130705 1
<h4>Objective</h4>To use an empirical Bayesian approach, blending practice, and group quality data with physician results to increase the accuracy of quality of care measures.<h4>Data sources</h4>Performance data on diabetes glycemic screening for 8,357 physicians collected from multiple payers as part of a statewide physician performance reporting initiative.<h4>Study design</h4>A variance components analysis assessed the strength of group, practice, and physician effects compared with random e ...[more]
Similar Datasets
Project description:ObjectiveTo evaluate the comparability of commercially available practice site data from SK&A with survey data to understand the implications of using SK&A data for health services research.Data sourcesResponses to the Comprehensive Primary Care Plus (CPC+) Practice Survey and SK&A data.Study designComparison of CPC + Practice Survey responses to SK&A information for 2698 primary care practice sites.Data collectionCPC + Practice Survey data collected through a web-only survey from April through September 2017, and SK&A data purchased in November 2016.Principal findingsInformation was similar across data sources, although some discrepancies were common. For example, 56% of practice sites had differences in the reported number of practitioners, and larger sites tended to have larger differences. Among practice sites with 1 practitioner in the survey, only 1.3% had a difference of 3 or more practitioners between the data sources, whereas 63% of practice sites with 11 or more practitioners had a difference of 3 or more practitioners.ConclusionsDiscrepancies between data sources could reflect differences of interpretation when defining practice site characteristics, changes over time in those characteristics, or data errors in either SK&A or the survey. Researchers using SK&A data should consider possible ramifications for their studies.
| S-EPMC7969202 | biostudies-literature
Project description:Research objectiveTo assess the extent to which the organizational culture of physician group practices is associated with individual physician satisfaction with the managerial and organizational capabilities of the groups.Study design and methodsPhysician surveys from 1997 to 1998 assessing the culture of their medical groups and their satisfaction with six aspects of group practice. Organizational culture was conceptualized using the Competing Values framework, yielding four distinct cultural types. Physician-level data were aggregated to the group level to attain measures of organizational culture. Using hierarchical linear modeling, individual physician satisfaction with six dimensions of group practice was predicted using physician-level variables and group-level variables. Separate models for each of the four cultural types were estimated for each of the six satisfaction measures, yielding a total of 24 models. SAMPLE STUDIED: Fifty-two medical groups affiliated with 12 integrated health systems from across the U.S., involving 1,593 physician respondents (38.3 percent response rate). Larger medical groups and multispecialty groups were over-represented compared with the U.S. as a whole.Principal findingsOur models explain up to 31 percent of the variance in individual physician satisfaction with group practice, with individual organizational culture scales explaining up to 5 percent of the variance. Group-level predictors: group (i.e., participatory) culture was positively associated with satisfaction with staff and human resources, technological sophistication, and price competition. Hierarchical (i.e., bureaucratic) culture was negatively associated with satisfaction with managerial decision making, practice level competitiveness, price competition, and financial capabilities. Rational (i.e., task-oriented) culture was negatively associated with satisfaction with staff and human resources, and price competition. Developmental (i.e., risk-taking) culture was not significantly associated with any of the satisfaction measures. In some of the models, being a single-specialty group (compared with a primary care group) and a group having a higher percent of male physicians were positively associated with satisfaction with financial capabilities. Physician-level predictors: individual physicians' ratings of organizational culture were significantly related to many of the satisfaction measures. In general, older physicians were more satisfied than younger physicians with many of the satisfaction measures. Male physicians were less satisfied with data capabilities. Primary care physicians (versus specialists) were less satisfied with price competition.ConclusionSome dimensions of physician organizational culture are significantly associated with various aspects of individual physician satisfaction with group practice.
| S-EPMC1955261 | biostudies-literature
Project description:ContextThe Centers for Medicare & Medicaid Services (CMS) recently launched accountable care organization (ACO) programs designed to improve quality and slow cost growth. The ACOs resemble an earlier pilot, the Medicare Physician Group Practice Demonstration (PGPD), in which participating physician groups received bonus payments if they achieved lower cost growth than local controls and met quality targets. Although evidence indicates the PGPD improved quality, uncertainty remains about its effect on costs.ObjectiveTo estimate cost savings associated with the PGPD overall and for beneficiaries dually eligible for Medicare and Medicaid.DesignQuasi-experimental analyses comparing preintervention (2001-2004) and postintervention (2005-2009) trends in spending of PGPD participants to local control groups. We compared estimates using several alternative approaches to adjust for case mix.SettingTen physician groups from across the United States.Patients and participantsThe intervention group was composed of fee-for-service Medicare beneficiaries (n = 990,177) receiving care primarily from the physicians in the participating medical groups. Controls were Medicare beneficiaries (n = 7,514,453) from the same regions who received care largely from non-PGPD physicians. Overall, 15% of beneficiaries were dually eligible for Medicare and Medicaid.Main outcome measureAnnual spending per Medicare fee-for-service beneficiary.ResultsAnnual savings per beneficiary were modest overall (adjusted mean $114, 95% CI, $12-$216). Annual savings were significant in dually eligible beneficiaries (adjusted mean $532, 95% CI, $277-$786), but were not significant among nondually eligible beneficiaries (adjusted mean $59, 95% CI, $166 in savings to $47 in additional spending). The adjusted mean spending reductions were concentrated in acute care (overall, $118, 95% CI, $65-$170; dually eligible: $381, 95% CI, $247-$515; nondually eligible: $85, 95% CI, $32-$138). There was significant variation in savings across practice groups, ranging from an overall mean per-capita annual saving of $866 (95% CI, $815-$918) to an increase in expenditures of $749 (95% CI, $698-$799). Thirty-day medical readmissions decreased overall (-0.67%, 95% CI, -1.11% to -0.23%) and in the dually eligible (-1.07%, 95% CI, -1.73% to -0.41%), while surgical readmissions decreased only for the dually eligible (-2.21%, 95% CI, -3.07% to -1.34%). Estimates were sensitive to the risk-adjustment method.ConclusionsSubstantial PGPD savings achieved by some participating institutions were offset by a lack of saving at other participating institutions. Most of the savings were concentrated among dually eligible beneficiaries.
| S-EPMC3484377 | biostudies-literature
Project description:BackgroundQuality of cardiovascular disease (CVD) preventive care is suboptimal. Recent data correlated increasing years in practice for physicians with lower-quality health care.ObjectiveThe purpose of this study was to assess physician awareness/adherence to national blood pressure, cholesterol, and CVD prevention guidelines for women according to physician/practice characteristics.DesignStandardized online survey and experimental case studies were administered to 500 randomly selected U.S. physicians. Multivariable regression models tested physician age, gender, specialty, and practice type as independent predictors of guideline awareness/adherence.ResultsCompared with older physicians (50+ years), younger physicians (<50 years) reported a lower level of awareness of cholesterol guidelines (P=.04) and lower incorporation of women's guidelines (P=.02). Yet, older physicians were less likely to recommend weight management for high-risk cases (P=.03) and less confident in helping patients manage weight (P=.045) than younger physicians. Older physicians were also less likely to identify a low-density lipoprotein<100 mg/dL as optimal versus younger physicians (P=.01), as were solo versus nonsolo practitioners (P=.02). Solo practitioners were less aware of cholesterol guidelines (P=.04) and were more likely to prescribe aspirin for low-risk female patients than nonsolo practitioners (P<.01). Solo practitioners rated their clinical judgment as more effective than guidelines in improving patient health outcomes (P<.01) and more frequently rated the patient as the greatest barrier to CVD prevention versus nonsolo practitioners (P<.01).ConclusionsThough guideline awareness is high, efforts to promote their utilization are needed and may improve quality outcomes. Targeted education and support for CVD prevention may be helpful to older and solo physicians.
| S-EPMC1828095 | biostudies-literature
Project description:BackgroundAlthough response rates for physician surveys have been decreasing, it is not clear whether this trend is associated with an increase in survey nonresponse bias. One means for assessing potential bias is to conduct a level-of-effort analysis that compares data estimates for respondents interviewed during the first recruitment contact to respondents interviewed at later recontact cycles.MethodsWe compared early and later responders within the Survey of Physician Attitudes Regarding the Care of Cancer Survivors with respect to both demographic characteristics and aggregate survey responses to items on survivor care knowledge, attitudes, and practices.ResultsAccumulating additional completions across each of 4 respondent contact attempts improved weighted response rates (35.0%, 46.9%, 52.3%, and 57.6%, respectively). However, the majority of estimates for analyzed variables remained relatively unchanged over additional cycles of recontact.ConclusionsWe conclude that additional respondent recontact attempts, especially beyond a single recontact, had little influence on key data distributions, suggesting that these were ineffective in reducing nonresponse bias. Further, the conduct of additional recruitment recontacts was an inefficient means for increasing statistical power. For the conduct of physician surveys, a practice that may in some cases be cost-effective, while also controlling total survey error, is to establish a larger initial sample; to either eliminate nonresponse follow-up or to limit this to one recontact; and to accept a somewhat lower final overall survey response rate.
| S-EPMC3784997 | biostudies-literature
Project description:BackgroundMedical group practices are central to many of the proposals for health care reform, but little is known about the relationship between practice-level characteristics and the quality and cost of care.MethodsPractice characteristics from a 2009 national survey of 211 group practices were linked to Medicare claims data for beneficiaries attributed to the practices. Multivariate regression was used to examine the relationship between practice characteristics and claims-computable measures of screening and monitoring, avoidable utilization, risk-adjusted per-beneficiary per-year (PBPY) costs, and the practice's net revenue.ResultsSeveral characteristics of group practices are predictive of screening and monitoring measures. Those measures, in turn, are predictive of lower values of avoidable utilization measures that contribute to higher PBPY costs. The effects of group practice characteristics on avoidable utilization, cost, and practice net revenue appear to work primarily through improved screening and monitoring.ConclusionsPractice characteristics influence costs indirectly through a set of statistically significant relationships among screening and monitoring measures and avoidable utilization. However, these relationships are not the only pathways connecting practice characteristics to cost and those additional pathways contain substantial "noise" adding uncertainty to the estimated direct effects. Some of the attributes thought to be important characteristics of accountable care organizations and medical homes appear to be associated with lower quality and no improvement in cost.
| S-EPMC4450926 | biostudies-literature
Project description:BackgroundChatGPT-4 is the latest release of a novel artificial intelligence (AI) chatbot able to answer freely formulated and complex questions. In the near future, ChatGPT could become the new standard for health care professionals and patients to access medical information. However, little is known about the quality of medical information provided by the AI.ObjectiveWe aimed to assess the reliability of medical information provided by ChatGPT.MethodsMedical information provided by ChatGPT-4 on the 5 hepato-pancreatico-biliary (HPB) conditions with the highest global disease burden was measured with the Ensuring Quality Information for Patients (EQIP) tool. The EQIP tool is used to measure the quality of internet-available information and consists of 36 items that are divided into 3 subsections. In addition, 5 guideline recommendations per analyzed condition were rephrased as questions and input to ChatGPT, and agreement between the guidelines and the AI answer was measured by 2 authors independently. All queries were repeated 3 times to measure the internal consistency of ChatGPT.ResultsFive conditions were identified (gallstone disease, pancreatitis, liver cirrhosis, pancreatic cancer, and hepatocellular carcinoma). The median EQIP score across all conditions was 16 (IQR 14.5-18) for the total of 36 items. Divided by subsection, median scores for content, identification, and structure data were 10 (IQR 9.5-12.5), 1 (IQR 1-1), and 4 (IQR 4-5), respectively. Agreement between guideline recommendations and answers provided by ChatGPT was 60% (15/25). Interrater agreement as measured by the Fleiss κ was 0.78 (P<.001), indicating substantial agreement. Internal consistency of the answers provided by ChatGPT was 100%.ConclusionsChatGPT provides medical information of comparable quality to available static internet information. Although currently of limited quality, large language models could become the future standard for patients and health care professionals to gather medical information.
| S-EPMC10365578 | biostudies-literature
Project description:ObjectiveTo estimate the joint effect of a multifaceted access intervention on primary care physician (PCP) productivity in a large, integrated prepaid group practice.Data sourcesAdministrative records of physician characteristics, compensation and full-time equivalent (FTE) data, linked to enrollee utilization and cost information.Study designDependent measures per quarter per FTE were office visits, work relative value units (WRVUs), WRVUs per visit, panel size, and total cost per member per quarter (PMPQ), for PCPs employed >0.25 FTE. General estimating equation regression models were included provider and enrollee characteristics.Principal findingsPanel size and RVUs per visit rose, while visits per FTE and PMPQ cost declined significantly between baseline and full implementation. Panel size rose and visits per FTE declined from baseline through rollout and full implementation. RVUs per visit and RVUs per FTE first declined, and then increased, for a significant net increase of RVUs per visit and an insignificant rise in RVUs per FTE between baseline and full implementation. PMPQ cost rose between baseline and rollout and then declined, for a significant overall decline between baseline and full implementation.ConclusionsThis organization-wide access intervention was associated with improvements in several dimensions in PCP productivity and gains in clinical efficiency.
| S-EPMC2654163 | biostudies-literature
Project description:BackgroundClinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats - open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) - in rheumatologists' pre-treatment assessment for tumor-necrosis-factor (TNF) blocker therapy.MethodsStudy designProspective randomized study.SettingRheumatologists attending the 2004 French Society of Rheumatology meeting. Physicians were given a vignette describing the history of a fictitious woman with active rheumatoid arthritis, who was a candidate for therapy with TNF blocking agents, and then were randomized to receive questionnaire A, B, or C, each containing the same four questions but with different response formats, that asked about their pretreatment assessment.MeasurementsLong (recommended items) and short (mandatory items) checklists were developed for pretreatment assessment for TNF-blocker therapy, and scores were expressed on the basis of responses to questionnaires A, B, and C as the percentage of respondents correctly choosing explicit items on these checklists.Statistical analysisComparison of the selected items using pairwise Chi-square tests with Bonferonni correction for variables with statistically significant differences.ResultsData for all surveys distributed (114 As, 118 Bs, and 118 Cs) were complete and available for analysis. The percentage of questionnaire A, B, and C respondents for whom data was correctly complete for the short checklist was 50.4%, 84.0% and 95.0%, respectively, and was 0%, 5.0% and 5.9%, respectively, for the long version. As an example, 65.8%, 85.7% and 95.8% of the respondents of A, B, and C questionnaires, respectively, correctly identified the need for tuberculin skin test (p < 0.0001).ConclusionIn evaluating clinical practice with use of a clinical vignette, a multiple-choice format rather than an open-ended format overestimates physician performance. The insertion of deceptive response items mixed with correct items in closed-ended (multiple-choice) questionnaire failed to avoid this overestimation.
| S-EPMC3224732 | biostudies-literature
Project description:IntroductionYouTube is the second most popular search website worldwide to access health information online. This study was undertaken to assess the reliability and quality of information about myositis on YouTube and delineate attributes of useful videos using standard metrics.MethodsWe conducted a thorough search on YouTube using 9 search terms related to myositis. The inclusion criteria were content related to myositis, English language and acceptable audio-video quality. Duplicates and advertisements were excluded from the analysis. Videos were classified as useful, not very useful or misleading and patient narratives. Reliability was determined using the mDISCERN criteria, quality using the Global Quality Scale (GQS) and JAMA system, using appropriate cut-offs (mDISCERN > 4, GQS > 4, JAMA > 3).ResultsOut of a total of 900 videos, 453 were included for the analysis. Seventy-four per cent and 2% provided useful and not very useful information respectively, while 24% were patient narratives. Seventy-one per cent were intended specifically for patients while 69% were for healthcare providers and students. Noteworthily, useful and not very useful videos had similar total views though the number of likes and daily viewership were higher for useful videos (p = 0.024, p = 0.046). Nearly half (47%) of useful videos were by professional medical societies/patient support groups (PSGs) while not very useful ones were by nonmedical media (38%). Physician-predicted usefulness was discordant with score-based usefulness (κ = 0.129). However, GQS emerged as a significant (p = 0.008) predictor of video usefulness in multivariate analysis.ConclusionA large majority of English YouTube videos on myositis provide useful information for patients. Physicians could signpost patients to high-quality useful videos as determined by GQS and sources like professional medical societies and PSGs. Key Points •This study highlights the importance of regulating health information posted online, accessed by millions of people, to gauge the quality of information and to identify and curb misinformation. •It also identifies recommendations for the future for uploading such content on the Internet. •The implications lie in our patients being better informed about their disease as they are important stakeholders in the healthcare decision-making process.
| S-EPMC9910767 | biostudies-literature