Variance components of ratings of physician-patient communication: A generalizability theory analysis.
Ontology highlight
ABSTRACT: Background
The ratings of physician-patient communication are an important indicator of the quality of health care delivery and provide guidance for many important decisions in the health care setting and in health research. But there is no gold standard to assess physician-patient communication. Thus, depending on the specific measurement condition, multiple sources of variance may contribute to the total score variance of ratings of physician-patient communication. This may systematically impair the validity of conclusions drawn from rating data.Objective
To examine the extent to which different measurement conditions and rater perspectives, respectively contribute to the variance of physician-patient communication ratings.Methods
Variance components of ratings of physician-patient communication gained from 32 general practitioners and 252 patients from 25 family practices in Germany were analyzed using generalizability theory. The communication dimensions "shared decision making", "effective and open communication" and "satisfaction" were considered.Results
Physician-patient communication ratings most substantially reflect unique rater-perspective and communication dimension combinations (32.7% interaction effect). The ratings also represented unique physician and rater-perspective combinations (16.3% interaction effect). However, physicians' communication behavior and the observed communication dimensions revealed only a low extent of score variance (1% physician effect; 3.7% communication dimension effect). Approximately half of the variance remained unexplained (46.2% three-way interaction, confounded with error).Conclusion
The ratings of physician-patient communication minimally reflect physician communication skills in general. Instead, these ratings exhibit primarily differences among physicians and patients in their tendency to perceive shared decision making and effective and open communication and to be satisfied with communication, regardless of the communication behavior of physicians. Rater training and assessing low inferential ratings of physician-patient communication dimensions should be considered when subjective aspects of rater perspectives are not of interest.
SUBMITTER: Rottele N
PROVIDER: S-EPMC8191893 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundInformation from ratings sites are increasingly informing patient decisions related to health care and the selection of physicians.ObjectiveThe current study sought to determine the validity of online patient ratings of physicians through comparison with physician peer review.MethodsWe extracted 223,715 reviews of 41,104 physicians from 10 of the largest cities in the United States, including 1142 physicians listed as "America's Top Doctors" through physician peer review. Differences in mean online patient ratings were tested for physicians who were listed and those who were not.ResultsOverall, no differences were found between the online patient ratings based upon physician peer review status. However, statistical differences were found for four specialties (family medicine, allergists, internal medicine, and pediatrics), with online patient ratings significantly higher for those physicians listed as a peer-reviewed "Top Doctor" versus those who were not.ConclusionsThe results of this large-scale study indicate that while online patient ratings are consistent with physician peer review for four nonsurgical, primarily in-office specializations, patient ratings were not consistent with physician peer review for specializations like anesthesiology. This result indicates that the validity of patient ratings varies by medical specialization.
| S-EPMC5913572 | biostudies-literature
Project description:BackgroundResidents may benefit from simulated practice with personalized feedback to prepare for high-stakes disclosure conversations with patients after harmful errors and to meet American Council on Graduate Medical Education mandates. Ideally, feedback would come from patients who have experienced communication after medical harm, but medical researchers and leaders have found it difficult to reach this community, which has made this approach impractical at scale. The Video-Based Communication Assessment app is designed to engage crowdsourced laypeople to rate physician communication skills but has not been evaluated for use with medical harm scenarios.ObjectiveWe aimed to compare the reliability of 2 assessment groups (crowdsourced laypeople and patient advocates) in rating physician error disclosure communication skills using the Video-Based Communication Assessment app.MethodsInternal medicine residents used the Video-Based Communication Assessment app; the case, which consisted of 3 sequential vignettes, depicted a delayed diagnosis of breast cancer. Panels of patient advocates who have experienced harmful medical error, either personally or through a family member, and crowdsourced laypeople used a 5-point scale to rate the residents' error disclosure communication skills (6 items) based on audiorecorded responses. Ratings were aggregated across items and vignettes to create a numerical communication score for each physician. We used analysis of variance, to compare stringency, and Pearson correlation between patient advocates and laypeople, to identify whether rank order would be preserved between groups. We used generalizability theory to examine the difference in assessment reliability between patient advocates and laypeople.ResultsInternal medicine residents (n=20) used the Video-Based Communication Assessment app. All patient advocates (n=8) and 42 of 59 crowdsourced laypeople who had been recruited provided complete, high-quality ratings. Patient advocates rated communication more stringently than crowdsourced laypeople (patient advocates: mean 3.19, SD 0.55; laypeople: mean 3.55, SD 0.40; P<.001), but patient advocates' and crowdsourced laypeople's ratings of physicians were highly correlated (r=0.82, P<.001). Reliability for 8 raters and 6 vignettes was acceptable (patient advocates: G coefficient 0.82; crowdsourced laypeople: G coefficient 0.65). Decision studies estimated that 12 crowdsourced layperson raters and 9 vignettes would yield an acceptable G coefficient of 0.75.ConclusionsCrowdsourced laypeople may represent a sustainable source of reliable assessments of physician error disclosure skills. For a simulated case involving delayed diagnosis of breast cancer, laypeople correctly identified high and low performers. However, at least 12 raters and 9 vignettes are required to ensure adequate reliability and future studies are warranted. Crowdsourced laypeople rate less stringently than raters who have experienced harm. Future research should examine the value of the Video-Based Communication Assessment app for formative assessment, summative assessment, and just-in-time coaching of error disclosure communication skills.
| S-EPMC9107044 | biostudies-literature
Project description:To determine the association of patients' ratings of the patient-physician relationship with physicians' self-reported cultural competence (CC).Physicians completed a survey assessing their CC in three domains: motivation to learn about other cultures (motivation attitudes), awareness of white privilege and acceptance of a racial group's choice to retain distinct customs and values (power assimilation attitudes), and clinical behaviors reflective of CC. Their African-American and white patients completed interviews assessing satisfaction with the medical visit, trust in their physician, perceptions of their physician's respect for them and their participation in care. We conducted regression analyses to explore the associations between CC and patient ratings of the relationship.Patients of physicians reporting more motivation to learn about other cultures were more satisfied (OR = 2.1, 95% CI = 1.0-4.4), perceived their physicians were more facilitative (beta = 0.4, p = 0.02) and reported seeking and sharing more information during the medical visit (beta = 0.2, p = 0.03). Physicians' power assimilation attitudes were associated with patients' ratings of physician facilitation (beta = 0.4, p = 0.02). Patients of physicians reporting more frequent CC behaviors were more satisfied (OR = 3.1, 95% CI = 1.4-6.9) and reported seeking and sharing more information (beta = 0.3, p = 0.04).Attitudinal and behavioral components of CC are important to developing higher quality, participative relationships between patients and their physicians.
| S-EPMC2659158 | biostudies-other
Project description:One factor that affects the reliability of observed scores is restriction of range on the construct measured for a particular group of study participants. This study illustrates how researchers can use generalizability theory to evaluate the impact of restriction of range in particular sample characteristics on the generalizability of test scores and to estimate how changes in measurement design could improve the generalizability of the test scores. An observer-rated measure of child self-regulation (Response to Challenge Scale; Lakes, 2011) is used to examine scores for 198 children (Grades K through 5) within the generalizability theory (GT) framework. The generalizability of ratings within relatively developmentally homogeneous samples is examined and illustrates the effect of reduced variance among ratees on generalizability. Forecasts for g coefficients of various D study designs demonstrate how higher generalizability could be achieved by increasing the number of raters or items. In summary, the research presented illustrates the importance of and procedures for evaluating the generalizability of a set of scores in a particular research context.
| S-EPMC3657319 | biostudies-literature
Project description:Numerous empirical studies from various populations and settings link patient treatment adherence to physician-patient communication. Meta-analysis allows estimates of the overall effects both in correlational research and in experimental interventions involving the training of physicians' communication skills.Calculation and analysis of "r effect sizes" and moderators of the relationship between physician's communication and patient adherence, and the effects of communication training on adherence to treatment regimens for varying medical conditions.Thorough search of published literature (1949-August 2008) producing separate effects from 106 correlational studies and 21 experimental interventions. Determination of random effects model statistics and the detailed examination of study variability using moderator analyses.Physician communication is significantly positively correlated with patient adherence; there is a 19% higher risk of non-adherence among patients whose physician communicates poorly than among patients whose physician communicates well. Training physicians in communication skills results in substantial and significant improvements in patient adherence such that with physician communication training, the odds of patient adherence are 1.62 times higher than when a physician receives no training.Communication in medical care is highly correlated with better patient adherence, and training physicians to communicate better enhances their patients' adherence. Findings can contribute to medical education and to interventions to improve adherence, supporting arguments that communication is important and resources devoted to improving it are worth investing in. Communication is thus an important factor over which physicians have some control in helping their patients to adhere.
| S-EPMC2728700 | biostudies-literature
Project description:RationalePatient-physician communication plays an essential role in a variety of patient outcomes; however, it is often difficult to operationalize positive patient-physician communication objectively, and the existing evaluation tools are generally time-consuming.ObjectiveThis study proposes semantic similarity of the patient's and physician's language in a medical interaction as a measure of patient-physician communication. Latent semantic analysis (LSA), a mathematical method for modeling semantic meaning, was employed to assess similarity in language during clinical interactions between physicians and patients.MethodsParticipants were 132 Black/African American patients (76% women, Mage = 43.8, range = 18-82) who participated in clinical interactions with 17 physicians (53% women, Mage = 27.1, range = 26-35) in a primary care clinic in a large city in the Midwestern United States.ResultsLSA captured reliable information about patient-physician communication: The mean correlation indicating similarity between the transcripts of a physician and patient in a clinical interaction was 0.142, significantly greater than zero; the mean correlation between a patient's transcript and transcripts of their physician during interactions with other patients was not different from zero. Physicians differed significantly in the semantic similarity between their language and that of their patients, and these differences were related to physician ethnicity and gender. Female patients exhibited greater communication similarity with their physicians than did male patients. Finally, greater communication similarity was predicted by less patient trust in physicians prior to the interaction and greater patient trust after the interaction.ConclusionLSA is a potentially important tool in patient-physician communication research. Methodological considerations in applying LSA to address research questions in patient-physician communication are discussed.
| S-EPMC5866755 | biostudies-literature
Project description:ImportanceEvaluation of trainees in graduate medical education training programs using Milestones has been in place since 2013. It is not known whether trainees who have lower ratings during the last year of training go on to have concerns related to interactions with patients in posttraining practice.ObjectiveTo investigate the association between resident Milestone ratings and posttraining patient complaints.Design, setting, and participantsThis retrospective cohort study included physicians who completed Accreditation Council for Graduate Medical Education (ACGME)-accredited programs between July 1, 2015, and June 30, 2019, and worked at a site that participated in the national Patient Advocacy Reporting System (PARS) program for at least 1 year. Milestone ratings from ACGME training programs and patient complaint data from PARS were collected. Data analysis was conducted from March 2022 to February 2023.ExposuresLowest professionalism (P) and interpersonal and communication skills (ICS) Milestones ratings 6 months prior to the end of training.Main outcomes and measuresPARS year 1 index scores, based on recency and severity of complaints.ResultsThe cohort included 9340 physicians with median (IQR) age of 33 (31-35) years; 4516 (48.4%) were women physicians. Overall, 7001 (75.0%) had a PARS year 1 index score of 0, 2023 (21.7%) had a score of 1 to 20 (moderate), and 316 (3.4%) had a score of 21 or greater (high). Among physicians in the lowest Milestones group, 34 of 716 (4.7%) had high PARS year 1 index scores, while 105 of 3617 (2.9%) with Milestone ratings of 4.0 (proficient), had high PARS year 1 index scores. In a multivariable ordinal regression model, physicians in the 2 lowest Milestones rating groups (0-2.5 and 3.0-3.5) were statistically significantly more likely to have higher PARS year 1 index scores than the reference group with Milestones ratings of 4.0 (0-2.5 group: odds ratio, 1.2 [95% CI, 1.0-1.5]; 3.0-3.5 group: odds ratio, 1.2 [95% CI, 1.1-1.3]).Conclusions and relevanceIn this study, trainees with low Milestone ratings in P and ICS near the end of residency were at increased risk for patient complaints in their early posttraining independent physician practice. Trainees with lower Milestone ratings in P and ICS may need more support during graduate medical education training or in the early part of their posttraining practice career.
| S-EPMC10091163 | biostudies-literature
Project description:Although previous studies suggest an association between provider burnout and suboptimal self-reported communication, no studies relate physician burnout to observed patient-physician communication behaviors.To investigate the relationship between physician burnout and observed patient-physician communication outcomes in patient-physician encounters.Longitudinal study of enrollment data from a trial of interventions to improve patient adherence to hypertension treatment.Fifteen urban community-based clinics in Baltimore, MD.Forty physicians and 235 of their adult hypertensive patients, with oversampling of ethnic minorities and poor persons. Fifty-three percent of physicians were women, and the average practice experience was 11.2 years. Among the 235 patients, 66% were women, 60% were African-American, and 90% were insured.Audiotape analysis of communication during outpatient encounters (one per patient) using the Roter Interaction Analysis System and patients' ratings of satisfaction with and trust and confidence in the physician.The median time between the physician burnout assessment and the patient encounter was 15.1 months (range 5.6-30). Multivariate analyses revealed no significant differences in physician communication based on physician burnout. However, compared with patients of low-burnout physicians, patients of high-burnout physicians gave twice as many negative rapport-building statements (incident risk ratio 2.06, 95% CI 1.58-2.86, p < 0.001). Physician burnout was not significantly associated with physician or patient affect, patient-centeredness, verbal dominance, or length of the encounter. Physician burnout was also not significantly associated with patients' ratings of their satisfaction, confidence, or trust.Physician burnout was not associated with physician communication behaviors nor with most measures of patient-centered communication. However, patients engaged in more rapport-building behaviors. These findings suggest a complex relationship between physician burnout and patient-physician communication, which should be investigated and linked to patient outcomes in future research.
| S-EPMC2533387 | biostudies-other
Project description:BackgroundExcellent communication is a necessary component of high-quality health care. We aimed to determine whether a training module could improve patients' perceptions of physician communication behaviors, as measured by change over time in domains of patient experience scores related to physician communication.Study designWe designed a comprehensive physician-training module focused on improving specific "etiquette-based" physician communication skills through standardized simulations and physician coaching with structured feedback. We employed a quasi-experimental pre-post design, with an intervention group consisting of internal medicine hospitalists and residents and a control group consisting of surgeons. The outcome was percent "always" scores for questions related to patients' perceptions of physician communication using the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey and a Non-HCAHPS Physician-Specific Patient Experience Survey (NHPPES) administered to patients cared for by hospitalists.ResultsA total of 128 physicians participated in the simulation. Responses from 5020 patients were analyzed using HCAHPS survey data and 1990 patients using NHPPES survey data. The intercept shift, or the degree of change from pre-intervention percent "always" responses, for the HCAHPS questions of doctors "treating patients with courtesy" "explaining things in a way patients could understand," and "overall teamwork" showed no significant differences between surgical control and hospitalist intervention patients. Adjusted NHPPES percent excellent survey results increased significantly post-intervention for the questions of specified individual doctors "keeping patient informed" (adjusted intercept shift 9.9% P = 0.019), "overall teamwork" (adjusted intercept shift 11%, P = 0.037), and "using words the patient could understand" (adjusted intercept shift 14.8%, p = 0.001).ConclusionA simulation based physician communication coaching method focused on specific "etiquette-based" communication behaviors through a deliberate practice framework was not associated with significantly improved HCAHPS physician communication patient experience scores. Further research could reveal ways that this model affects patients' perceptions of physician communication relating to specific physicians or behaviors.
| S-EPMC5497987 | biostudies-literature
Project description:Despite the documented benefits of colorectal cancer screening, patient participation rates remain low. Physician recommendation has been identified as a significant predictor of screening completion.The aim of this study is to investigate how primary care physicians perceive colorectal cancer screening communication tasks, as well as to explore the form and content of actual screening discussions.The research design includes a mailed physician survey and a separate observational study in a sample of videotaped medical encounters. PARTICIPANTS AND DATA SOURCES: The participants were 270 primary care physicians who completed a mailed questionnaire (57.9% response rate) and 18 physician-patient encounters that included discussions of colorectal cancer screening.The questionnaire focused on perceived importance and accomplishment of communication tasks relevant to colorectal cancer screening. Two of the authors reviewed transcripts of videotaped physician encounters to determine whether the same communication tasks assessed in the survey were accomplished. Interrater reliability was high across all of the mutually exclusive coding categories (Kappa > .90).Physicians rated colonoscopy as the most important screening option to discuss; self-reports indicate that colonoscopy (84.8%) is more frequently mentioned than fecal occult blood test (FOBT; 49.4%), flexible sigmoidoscopy (34.1%), or computed tomography (CT) imaging (18.1%). Explaining benefits and risks, describing test procedure and frequency, eliciting patient preferences, and making a plan for screening were all viewed as very important. Self-reported accomplishment of these communication tasks was considerably higher than that observed in our separate videotape sample.Most physicians recognize and espouse the importance of recommending colorectal cancer screening to eligible patients. However, findings from both the physician survey and observational study suggest that physicians tend to overestimate the extent of discussions about screening. Interventions may be warranted to improve clinical practice.
| S-EPMC2219805 | biostudies-other