Dataset Information

Online physician ratings fail to predict actual performance on measures of quality, value, and peer review.

ABSTRACT: Objective:Patients use online consumer ratings to identify high-performing physicians, but it is unclear if ratings are valid measures of clinical performance. We sought to determine whether online ratings of specialist physicians from 5 platforms predict quality of care, value of care, and peer-assessed physician performance. Materials and Methods:We conducted an observational study of 78 physicians representing 8 medical and surgical specialties. We assessed the association of consumer ratings with specialty-specific performance scores (metrics including adherence to Choosing Wisely measures, 30-day readmissions, length of stay, and adjusted cost of care), primary care physician peer-review scores, and administrator peer-review scores. Results:Across ratings platforms, multivariable models showed no significant association between mean consumer ratings and specialty-specific performance scores (?-coefficient range, -0.04, 0.04), primary care physician scores (?-coefficient range, -0.01, 0.3), and administrator scores (?-coefficient range, -0.2, 0.1). There was no association between ratings and score subdomains addressing quality or value-based care. Among physicians in the lowest quartile of specialty-specific performance scores, only 5%-32% had consumer ratings in the lowest quartile across platforms. Ratings were consistent across platforms; a physician's score on one platform significantly predicted his/her score on another in 5 of 10 comparisons. Discussion:Online ratings of specialist physicians do not predict objective measures of quality of care or peer assessment of clinical performance. Scores are consistent across platforms, suggesting that they jointly measure a latent construct that is unrelated to performance. Conclusion:Online consumer ratings should not be used in isolation to select physicians, given their poor association with clinical performance.

SUBMITTER: Daskivich TJ

PROVIDER: S-EPMC7646854 | biostudies-literature | 2018 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Online physician ratings fail to predict actual performance on measures of quality, value, and peer review.

Daskivich Timothy J TJ Houman Justin J Fuller Garth G Black Jeanne T JT Kim Hyung L HL Spiegel Brennan B

Journal of the American Medical Informatics Association : JAMIA 20180401 4

<h4>Objective</h4>Patients use online consumer ratings to identify high-performing physicians, but it is unclear if ratings are valid measures of clinical performance. We sought to determine whether online ratings of specialist physicians from 5 platforms predict quality of care, value of care, and peer-assessed physician performance.<h4>Materials and methods</h4>We conducted an observational study of 78 physicians representing 8 medical and surgical specialties. We assessed the association of c ...[more]

PMID: 29025145

Similar Datasets

Project description:Purpose of the articleDuring the Covid-19 pandemic, formative OSCE were transformed into online OSCE, and senior students (near peers) substituted experienced clinical teachers. The aims of the study were to evaluate quality of the feedbacks given by near peers during online OSCEs and explore the experience of near-peer feedback from both learner's and near peer's perspectives.Materials and methodsAll 2nd year medical students (n = 158) attended an online OSCE under the supervision of twelve senior medical students. Outcome measures were 1) students' perception of the quality of the feedback through an online survey (Likert 1-5); 2) objective assessment of the quality of the feedback focusing on both the process and the content using a feedback scale (Likert 1-5); 3) experience of near peer feedback in two different focus groups.ResultsOne hundred six medical students answered the questionnaire and had their feedback session videotaped. The mean perceived overall quality of senior students' overall feedback was 4.75 SD 0.52. They especially valued self-evaluation (mean 4.80 SD 0.67), balanced feedback (mean 4.93 SD 0.29) and provision of simulated patient's feedback (mean 4.97 SD 0.17). The overall objective assessment of the feedback quality was 3.73 SD 0.38: highly scored skills were subjectivity (mean 3.95 SD 1.12) and taking into account student's self-evaluation (mean 3.71 (SD 0.87). Senior students mainly addressed history taking issues (mean items 3.53 SD 2.37) and communication skills (mean items 4.89 SD 2.43) during feedback. Participants reported that near peer feedback was less stressful and more tailored to learning needs- challenges for senior students included to remain objective and to provide negative feedback.ConclusionIncreased involvement of near peers in teaching activities is strongly supported for formative OSCE and should be implemented in parallel even if experience teachers are again involved in such teaching activities. However, it requires training not only on feedback skills but also on the specific content of the formative OSCE.

Project description:BackgroundUrologists are increasingly using various forms of social media to promote their professional practice and attract patients. Currently, the association of social media on a urologists' practice is unknown.ObjectivesWe aimed to determine whether social media presence is associated with higher online physician ratings and surgical volume among California urologists.MethodsWe sampled 195 California urologists who were rated on the ProPublica Surgeon Scorecard website. We obtained information on professional use of online social media (Facebook, Instagram, Twitter, blog, and YouTube) in 2014 and defined social media presence as a binary variable (yes/no) for use of an individual platform or any platform. We collected data on online physician ratings across websites (Yelp, Healthgrades, Vitals, RateMD, and UCompareHealthcare) and calculated the mean physician ratings across all websites as an average weighted by the number of reviews. We then collected data on surgical volume for radical prostatectomy from the ProPublica Surgeon Scorecard website. We used multivariable linear regression to determine the association of social media presence with physician ratings and surgical volume.ResultsAmong our sample of 195 urologists, 62 (32%) were active on some form of social media. Social media presence on any platform was associated with a slightly higher mean physician rating (? coefficient: .3; 95% CI 0.03-0.5; P=.05). However, only YouTube was associated with higher physician ratings (? coefficient: .3; 95% CI 0.2-0.5; P=.04). Social media presence on YouTube was strongly associated with increased radical prostatectomy volume (? coefficient: 7.4; 95% CI 0.3-14.5; P=.04). Social media presence on any platform was associated with increased radical prostatectomy volume (? coefficient: 7.1; 95% CI -0.7 to 14.2; P=.05).ConclusionsUrologists' use of social media, especially YouTube, is associated with a modest increase in physician ratings and prostatectomy volume. Although a majority of urologists are not currently active on social media, patients may be more inclined to endorse and choose subspecialist urologists who post videos of their surgical technique.

Project description:BackgroundPrevious research internationally has only analyzed publicly available feedback on physician rating websites (PRWs). However, it appears that many PRWs are not publishing all the feedback they receive. Analysis of this rejected feedback could provide a better understanding of the types of feedback that are currently not published and whether this is appropriate.ObjectiveThe aim of this study was to examine (1) the number of patient feedback rejected from the Swiss PRW Medicosearch, (2) the evaluation tendencies of the rejected patient feedback, and (3) the types of issues raised in the rejected narrative comments.MethodsThe Swiss PRW Medicosearch provided all the feedback that had been rejected between September 16, 2008, and September 22, 2017. The feedback were analyzed and classified according to a theoretical categorization framework of physician-, staff-, and practice-related issues.ResultsBetween September 16, 2008, and September 22, 2017, Medicosearch rejected a total of 2352 patient feedback. The majority of feedback rejected (1754/2352, 74.6%) had narrative comments in the German language. However, 11.9% (279/2352) of the rejected feedback only provided a quantitative rating with no narrative comment. Overall, 25% (588/2352) of the rejected feedback were positive, 18.7% (440/2352) were neutral, and 56% (1316/2352) were negative. The average rating of the rejected feedback was 2.8 (SD 1.4). In total, 44 subcategories addressing the physician (n=20), staff (n=9), and practice (n=15) were identified. In total, 3804 distinct issues were identified within the 44 subcategories of the categorization framework; 75% (2854/3804) of the issues were related to the physician, 6.4% (242/3804) were related to the staff, and 18.6% (708/3804) were related to the practice. Frequently mentioned issues identified from the rejected feedback included (1) satisfaction with treatment (533/1903, 28%); (2) the overall assessment of the physician (392/1903, 20.6%); (3) recommending the physician (345/1903, 18.1%); (4) the physician's communication (261/1903, 13.7%); (5) the physician's caring attitude (220/1903, 11.6%); and (6) the physician's friendliness (203/1903, 10.6%).ConclusionsIt is unclear why the majority of the feedback were rejected. This is problematic and raises concerns that online patient feedback are being inappropriately manipulated. If online patient feedback is going to be collected, there needs to be clear policies and practices about how this is handled. It cannot be left to the whims of PRWs, who may have financial incentives to suppress negative feedback, to decide which feedback is or is not published online. Further research is needed to examine how many PRWs are using criteria for determining which feedback is published or not, what those criteria are, and what measures PRWs are using to address the manipulation of online patient feedback.

Project description:BackgroundPhysician ratings websites have emerged as a novel forum for consumers to comment on their health care experiences. Little is known about such ratings in Canada.ObjectiveWe investigated the scope and trends for specialty, geographic region, and time for online physician ratings in Canada using a national data source from the country's leading physician-rating website.MethodsThis observational retrospective study used online ratings data from Canadian physicians (January 2005-September 2013; N=640,603). For specialty, province, and year of rating, we assessed whether physicians were likely to be rated favorably by using the proportion of ratings greater than the overall median rating.ResultsIn total, 57,412 unique physicians had 640,603 individual ratings. Overall, ratings were positive (mean 3.9, SD 1.3). On average, each physician had 11.2 (SD 10.1) ratings. By comparing specialties with Canadian Institute of Health Information physician population numbers over our study period, we inferred that certain specialties (obstetrics and gynecology, family practice, surgery, and dermatology) were more commonly rated, whereas others (pathology, radiology, genetics, and anesthesia) were less represented. Ratings varied by specialty; cardiac surgery, nephrology, genetics, and radiology were more likely to be rated in the top 50th percentile, whereas addiction medicine, dermatology, neurology, and psychiatry were more often rated in the lower 50th percentile of ratings. Regarding geographic practice location, ratings were more likely to be favorable for physicians practicing in eastern provinces compared with western and central Canada. Regarding year, the absolute number of ratings peaked in 2007 before stabilizing and decreasing by 2013. Moreover, ratings were most likely to be positive in 2007 and again in 2013.ConclusionsPhysician-rating websites are a relatively novel source of provider-level patient satisfaction and are a valuable source of the patient experience. It is important to understand the breadth and scope of such ratings, particularly regarding specialty, geographic practice location, and changes over time.

Project description:ImportanceThe relative value of facial plastic surgeon personal and practice attributes is relevant to the broader health care system because of increasing out-of-pocket expenses to patients.ObjectiveTo determine the relative value of specific facial plastic surgeon personal and practice attributes available online from the perspective of patients.Design, setting, and participantsThis study consisted of an electronic survey sent to patients by email using choice-based conjoint analysis; surveys were sent between December 2015 and March 2016. Participants had agreed to join email registries to be sent email surveys and promotions at 3 private facial plastic and reconstructive surgery practices. The following surgeon personal and practice attributes and levels were compared: (1) outcome transparency (above average, average, not available); (2) surgical training affiliations (US News and World Reports rankings); (3) online rating site scores (2 [poor], 3, or 4 [excellent] stars); and (4) price ($1×, $2×, and $3× [× = $1500; average cost was set at $2×]).Main outcomes and measuresThe relative importance of outcome transparency, surgical training affiliations, online rating scores, and price to prospective patients.ResultsOverall, 291 patients participated for a completion rate of 68%. Outcome transparency was the most valued attribute (attribute utility range = 141; attribute importance = 35.2%). Price was the least valued attribute (attribute utility range = 58.59; attribute importance = 15.1%). Assuming top-tier affiliations and 4-star ratings, share of market (SOM) was 75.5% for surgeons with above-average outcome transparency priced at $3× compared with those surgeons with no outcomes available priced at $1×. Holding price constant at $2×, surgeons with middle-tier affiliations and 2-star online ratings but above average outcomes achieved 48.4% SOM when compared with those surgeons with top-tier affiliations and 4-star online ratings without available outcomes.Conclusions and relevanceFacial plastic surgery patients most value surgeons who publish outcomes. Moreover, they are willing to discount poor rating scores and lower-ranked institutional affiliations when outcome transparency is high. This study demonstrates that outcome transparency is crucial in facial plastic surgery markets.Level of evidenceNA.

Project description:ObjectiveTo examine the extent to which medical group and market factors are related to individual primary care physician (PCP) performance on patient experience measures.Data sourcesThis study employs Clinician and Group CAHPS survey data (n=105,663) from 2,099 adult PCPs belonging to 34 diverse medical groups across California. Medical group directors were interviewed to assess the magnitude and nature of financial incentives directed at individual physicians and the adoption of patient experience improvement strategies. Primary care services area (PCSA) data were used to characterize the market environment of physician practices.Study designWe used multilevel models to estimate the relationship between medical group and market factors and physician performance on each Clinician and Group CAHPS measure. Models statistically controlled for respondent characteristics and accounted for the clustering of respondents within physicians, physicians within medical groups, and medical groups within PCSAs using random effects.Principal findingsCompared with physicians belonging to independent practice associations, physicians belonging to integrated medical groups had better performance on the communication ( p=.007) and care coordination ( p=.03) measures. Physicians belonging to medical groups with greater numbers of PCPs had better performance on all measures. The use of patient experience improvement strategies was not associated with performance. Greater emphasis on productivity and efficiency criteria in individual physician financial incentive formulae was associated with worse access to care ( p=.04). Physicians located in PCSAs with higher area-level deprivation had worse performance on the access to care ( p=.04) and care coordination ( p<.001) measures.ConclusionsPhysicians from integrated medical groups and groups with greater numbers of PCPs performed better on several patient experience measures, suggesting that organized care processes adopted by these groups may enhance patients' experiences. Physicians practicing in markets with high concentrations of vulnerable populations may be disadvantaged by constraints that affect performance. Future studies should clarify the extent to which performance deficits associated with area-level deprivation are modifiable.

Dataset Information

Online physician ratings fail to predict actual performance on measures of quality, value, and peer review.

Publications

Online physician ratings fail to predict actual performance on measures of quality, value, and peer review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure