Rating Communication in GP Consultations: The Association Between Ratings Made by Patients and Trained Clinical Raters.
Ontology highlight
ABSTRACT:
SUBMITTER: Burt J
PROVIDER: S-EPMC5858640 | biostudies-literature | 2018 Apr
REPOSITORIES: biostudies-literature
Publications
Rating Communication in GP Consultations: The Association Between Ratings Made by Patients and Trained Clinical Raters.
Burt Jenni J Abel Gary G Elmore Natasha N Newbould Jenny J Davey Antoinette A Llanwarne Nadia N Maramba Inocencio I Paddison Charlotte C Benson John J Silverman Jonathan J Elliott Marc N MN Campbell John J Roland Martin M
Medical care research and review : MCRR 20161003 2
Patient evaluations of physician communication are widely used, but we know little about how these relate to professionally agreed norms of communication quality. We report an investigation into the association between patient assessments of communication quality and an observer-rated measure of communication competence. Consent was obtained to video record consultations with Family Practitioners in England, following which patients rated the physician's communication skills. A sample of consult ...[more]
Similar Datasets
Project description:BackgroundWe consider the problem of assessing inter-rater agreement when there are missing data and a large number of raters. Previous studies have shown only 'moderate' agreement between pathologists in grading breast cancer tumour specimens. We analyse a large but incomplete data-set consisting of 24,177 grades, on a discrete 1-3 scale, provided by 732 pathologists for 52 samples.Methodology/principal findingsWe review existing methods for analysing inter-rater agreement for multiple raters and demonstrate two further methods. Firstly, we examine a simple non-chance-corrected agreement score based on the observed proportion of agreements with the consensus for each sample, which makes no allowance for missing data. Secondly, treating grades as lying on a continuous scale representing tumour severity, we use a Bayesian latent trait method to model cumulative probabilities of assigning grade values as functions of the severity and clarity of the tumour and of rater-specific parameters representing boundaries between grades 1-2 and 2-3. We simulate from the fitted model to estimate, for each rater, the probability of agreement with the majority. Both methods suggest that there are differences between raters in terms of rating behaviour, most often caused by consistent over- or under-estimation of the grade boundaries, and also considerable variability in the distribution of grades assigned to many individual samples. The Bayesian model addresses the tendency of the agreement score to be biased upwards for raters who, by chance, see a relatively 'easy' set of samples.Conclusions/significanceLatent trait models can be adapted to provide novel information about the nature of inter-rater agreement when the number of raters is large and there are missing data. In this large study there is substantial variability between pathologists and uncertainty in the identity of the 'true' grade of many of the breast cancer tumours, a fact often ignored in clinical studies.
| S-EPMC2488396 | biostudies-literature
Project description:PurposeInterpretation of screening tests such as mammograms usually require a radiologist's subjective visual assessment of images, often resulting in substantial discrepancies between radiologists' classifications of subjects' test results. In clinical screening studies to assess the strength of agreement between experts, multiple raters are often recruited to assess subjects' test results using an ordinal classification scale. However, using traditional measures of agreement in some studies is challenging because of the presence of many raters, the use of an ordinal classification scale, and unbalanced data.MethodsWe assess and compare the performances of existing measures of agreement and association as well as a newly developed model-based measure of agreement to three large-scale clinical screening studies involving many raters' ordinal classifications. We also conduct a simulation study to demonstrate the key properties of the summary measures.ResultsThe assessment of agreement and association varied according to the choice of summary measure. Some measures were influenced by the underlying prevalence of disease and raters' marginal distributions and/or were limited in use to balanced data sets where every rater classifies every subject. Our simulation study indicated that popular measures of agreement and association are prone to underlying disease prevalence.ConclusionsModel-based measures provide a flexible approach for calculating agreement and association and are robust to missing and unbalanced data as well as the underlying disease prevalence.
| S-EPMC5687310 | biostudies-literature
Project description:Variability between raters' ordinal scores is commonly observed in imaging tests, leading to uncertainty in the diagnostic process. In breast cancer screening, a radiologist visually interprets mammograms and MRIs, while skin diseases, Alzheimer's disease, and psychiatric conditions are graded based on clinical judgment. Consequently, studies are often conducted in clinical settings to investigate whether a new training tool can improve the interpretive performance of raters. In such studies, a large group of experts each classify a set of patients' test results on two separate occasions, before and after some form of training with the goal of assessing the impact of training on experts' paired ratings. However, due to the correlated nature of the ordinal ratings, few statistical approaches are available to measure association between raters' paired scores. Existing measures are restricted to assessing association at just one time point for a single screening test. We propose here a novel paired kappa to provide a summary measure of association between many raters' paired ordinal assessments of patients' test results before versus after rater training. Intrarater association also provides valuable insight into the consistency of ratings when raters view a patient's test results on two occasions with no intervention undertaken between viewings. In contrast to existing correlated measures, the proposed kappa is a measure that provides an overall evaluation of the association among multiple raters' scores from two time points and is robust to the underlying disease prevalence. We implement our proposed approach in two recent breast-imaging studies and conduct extensive simulation studies to evaluate properties and performance of our summary measure of association.
| S-EPMC7649720 | biostudies-literature
Project description:BackgroundGlobal coherence rating scales have been used by a number of researchers to examine spoken discourse in populations with and without acquired neurogenic communication disorders. The 4-point global coherence (GC) scale in the current study has demonstrated reliability and convergent validity. However, we have not yet established how a global coherence rating corresponds to functional communication.AimsThe current study explores the clinical meaningfulness of the 4-point GC rating scale. Survey questions and ratings were developed to examine discourse quality and functional coherence as perceived by untrained listeners. Ratings by untrained listeners were compared to trained discourse ratings using the established 4-point GC scale.MethodsTwelve discourse samples, scored by a trained rater, were selected for the current study from a previously collected set of discourse transcripts. Transcripts were extended monologues in response to one of four possible open-ended questions that were re-recorded by the trained rater to remove any distracting features of the original recordings, such as articulatory errors. Twenty-four untrained listeners rated the discourse samples using a short questionnaire that included questions for each sample about: topic maintenance, inclusion of unnecessary information, and the listener level of interest and attention. Questions for untrained listeners were based on operational definitions of global coherence and discourse quality respectively. These untrained ratings were compared to trained ratings of global coherence. Outcome measures were compared using non-parametric tests and a Spearman Rank Order test was also used to examine relationships among variables.Outcomes and resultsUntrained listeners' ratings for topic maintenance and inclusion of unnecessary information were significantly different between trained low, medium, and high global coherence ratings. Topic maintenance and inclusion of unnecessary information were also both significantly correlated with global coherence. Untrained listeners' ratings of their level of interest and attention for a sample were significantly different between trained medium-high and low-high global coherence ratings. Interest and attention ratings were also significantly correlated with ratings of global coherence.ConclusionsUntrained raters did differentiate between levels of global coherence using ratings of topic maintenance, inclusion of unnecessary information, and their level of attention and interest. Global coherence was also significantly correlated with the untrained ratings. These findings provide preliminary external validity for the global coherence scale and support its clinical utility.
| S-EPMC7500540 | biostudies-literature
Project description:BackgroundPrevious research internationally has only analyzed publicly available feedback on physician rating websites (PRWs). However, it appears that many PRWs are not publishing all the feedback they receive. Analysis of this rejected feedback could provide a better understanding of the types of feedback that are currently not published and whether this is appropriate.ObjectiveThe aim of this study was to examine (1) the number of patient feedback rejected from the Swiss PRW Medicosearch, (2) the evaluation tendencies of the rejected patient feedback, and (3) the types of issues raised in the rejected narrative comments.MethodsThe Swiss PRW Medicosearch provided all the feedback that had been rejected between September 16, 2008, and September 22, 2017. The feedback were analyzed and classified according to a theoretical categorization framework of physician-, staff-, and practice-related issues.ResultsBetween September 16, 2008, and September 22, 2017, Medicosearch rejected a total of 2352 patient feedback. The majority of feedback rejected (1754/2352, 74.6%) had narrative comments in the German language. However, 11.9% (279/2352) of the rejected feedback only provided a quantitative rating with no narrative comment. Overall, 25% (588/2352) of the rejected feedback were positive, 18.7% (440/2352) were neutral, and 56% (1316/2352) were negative. The average rating of the rejected feedback was 2.8 (SD 1.4). In total, 44 subcategories addressing the physician (n=20), staff (n=9), and practice (n=15) were identified. In total, 3804 distinct issues were identified within the 44 subcategories of the categorization framework; 75% (2854/3804) of the issues were related to the physician, 6.4% (242/3804) were related to the staff, and 18.6% (708/3804) were related to the practice. Frequently mentioned issues identified from the rejected feedback included (1) satisfaction with treatment (533/1903, 28%); (2) the overall assessment of the physician (392/1903, 20.6%); (3) recommending the physician (345/1903, 18.1%); (4) the physician's communication (261/1903, 13.7%); (5) the physician's caring attitude (220/1903, 11.6%); and (6) the physician's friendliness (203/1903, 10.6%).ConclusionsIt is unclear why the majority of the feedback were rejected. This is problematic and raises concerns that online patient feedback are being inappropriately manipulated. If online patient feedback is going to be collected, there needs to be clear policies and practices about how this is handled. It cannot be left to the whims of PRWs, who may have financial incentives to suppress negative feedback, to decide which feedback is or is not published online. Further research is needed to examine how many PRWs are using criteria for determining which feedback is published or not, what those criteria are, and what measures PRWs are using to address the manipulation of online patient feedback.
| S-EPMC7432139 | biostudies-literature
Project description:Cancer survivors often experience both short- and long-term challenges as a result of their cancer treatment. Many patients report feeling "lost in transition" following cancer treatment. The oncologist plays a key role in helping the patient to transition to survivorship. The purpose of this paper is to provide a descriptive understanding of the clinical care delivered during follow-up visits for lymphoma patients transitioning from active therapy into survivorship. We used thematic text analysis to identify themes in transcripts of audio recordings of 21 physicians and their patients in follow-up visits within 3 years of completing lymphoma treatment. Conversations between oncologists and patients during follow-up visits addressed a variety of themes, including specific health concerns, follow-up care, health promotion, prior treatment and test results, overall health status, emotional communication and affective tone, post-treatment rehabilitation, discussions occurring in the patient's physical examination, and social issues. However, the frequency with which these themes were discussed varied. In addition, there did not appear to be a consistent framework for these visits. The varied nature of the follow-up visits analyzed in this study suggests the need for increased structure in survivorship-care consultations.
| S-EPMC5617774 | biostudies-literature
Project description:This cross-sectional study evaluated and quantified the possible association of psychological symptoms and health status ratings on the burnout of healthcare workers from a tertiary medical center. Demographic data were obtained through a questionnaire survey. We evaluated their psychological symptoms using a 5-item Brief Symptom Rating Scale (BSRS-5) and burnout was measured using the Chinese version of the Maslach Burnout Inventory-Health Services Survey. The study began in Nov. 2018 and ended in Nov. 2020. A total of 2813 participants (men = 296, 10.5%; women = 2517, 89.5%) completed the questionnaires between December 2018 and March 2019. The response rate and complete rate were 71.7% and 93.2%. The multivariate analysis showed that, as the BSRS-5 level added, the odds ratio (OR) of burnout increased (BSRS-5 scores 6-9, 10-14, and 15-20; OR = 1.83, 3.23, and 9.15, separately; p value < 0.05). Overall, men (≥ 30 years of age) and women staffs with BSRS-5 scores ≥ 6; women with longer working hours (more than 46 h/week), men and women (≥ 30 years of age) working night shifts, and poor health status ratings were highly associated with burnout. The findings highlight the importance of screening for the BSRS-5 scores and health status ratings level for healthcare professionals at high risk of burnout, especially men ≥ 30 years of age and women with stressful working conditions.
| S-EPMC9062289 | biostudies-literature
Project description:BackgroundLittle information is available about whether the use of self-assessment instruments in primary care affects depression course and outcome. The purpose was to evaluate whether using a depression self-rating scale in recurrent person-centred GP consultations affected depression severity, quality of life, medication use, and sick leave frequency.MethodsPatients in the intervention group met their GP regularly at least 4 times during the 3 months intervention. In addition to treatment as usual (TAU), patients completed a self-assessment instrument (Montgomery-Asberg Depression Rating Scale) on each occasion, and then GPs used the completed instrument as the basis for a person-centred discussion of changes in depression symptoms. The control group received TAU. Frequency of visits in the TAU arm was the result of the GPs' and patients' joint assessments of care need in each case. Depression severity was measured with Beck Depression Inventory-II (BDI-II), quality of life with EQ-5D, and psychological well-being with the General Health Questionnaire-12 (GHQ-12). Data on sick leave, antidepressant and sedatives use, and care contacts were collected from electronic patient records. All variables were measured at baseline and 3, 6, and 12 months. Mean intra-individual changes were compared between the intervention and TAU group.ResultsThere were no significant differences between the intervention and control group in depression severity reduction or remission rate, change in quality of life, psychological well-being, sedative prescriptions, or sick leave during the whole 12-month follow-up. However, significantly more patients in the intervention group continued antidepressants until the 6 month follow-up (86/125 vs 78/133, p < 0.05).ConclusionsWhen GPs used a depression self-rating scale in recurrent consultations, patients more often continued antidepressant medication according to guidelines, compared to TAU patients. However, reduction of depressive symptoms, remission rate, quality of life, psychological well-being, sedative use, sick leave, and health care use 4-12 months was not significantly different from the TAU group. These findings suggest that frequent use of depression rating scales in person-centred primary care consultations has no further additional effect on patients' depression or well-being, sick leave, or health care use.Trial registrationClinicalTrials.gov Identifier: NCT01402206 . Registered June 27 2011(retrospectively registered).
| S-EPMC5244530 | biostudies-literature
Project description:BACKGROUND:It is unknown whether patients with medically unexplained symptoms (MUS) differ from patients with medically explained symptoms (MES) regarding their expectations and experiences on task-oriented communication (ie, communication in which the primary focus is on exchanging medical content), affect-oriented communication (ie, communication in which the primary focus is on the emotional aspects of the interaction) and therapy-oriented communication (ie, communication in which the primary focus is on therapeutic aspects) of the consultation and the extent to which GPs meet their expectations. OBJECTIVE:This study aims to explore (a) differences in patients' expectations and experiences in consultations with MUS patients and patients with MES and (b) the influence of patients' experiences in these consultations on their post-visit anxiety level. STUDY DESIGN:Prospective cohort. SETTING:Eleven Dutch general practices. MEASUREMENTS:Patients completed the QUOTE-COMM (Quality Of communication Through the patients' Eyes) questionnaire before and after the consultation to assess their expectations and experiences and these were related to changes in patients' state anxiety (abbreviated State-Trait Anxiety Inventory; STAI). RESULTS:Expectations did not differ between patients with MUS and MES. Patients presenting with either MUS or MES rated their experiences for task-related and affect-oriented communication of their GP higher than their expectations. GPs met patients' expectations less often on task-oriented communication in MUS patients compared to MES patients (70.2% vs 80.9%; P = ?0.001). Affect-oriented communication seems to be most important in reducing the anxiety level of MUS patients (? -0.63, 95% Cl = -1.07 to -0.19). DISCUSSION:Although the expectations of MUS patients are less often met compared to those of MES patients, GPs often communicate according to patients' expectations. Experiencing affect-oriented communication is associated with a stronger reduction in anxiety in patients with MUS than in those with MES. CONCLUSION:GPs communicate according to patients' expectations. However, GPs met patients' expectations on task-oriented communication less often in patients with MUS compared to patients with MES. Experiencing affect-oriented communication had a stronger association with the post-consultation anxiety for patients with MUS than MES.
| S-EPMC6543164 | biostudies-literature
Project description:BackgroundPhysician rating websites (PRWs) have been developed as part of a wider move toward transparency around health care quality, and these allow patients to anonymously rate, comment, and discuss physicians' quality on the Web. The first Swiss PRWs were established in 2008, at the same time as many international PRWs. However, there has been limited research conducted on PRWs in Switzerland to date. International research has indicated that a key shortcoming of PRWs is that they have an insufficient number of ratings.ObjectiveThe aim of this study was to examine the frequency of quantitative ratings and narrative comments on the Swiss PRWs.MethodsIn November 2017, a random stratified sample of 966 physicians was generated from the regions of Zürich and Geneva. Every selected physician was searched for on 4 rating websites (OkDoc, DocApp, Medicosearch, and Google) between November 2017 and July 2018. It was recorded whether the physician could be identified, what the physician's quantitative rating was, and whether the physician had received narrative comments. In addition, Alexa Internet was used to examine the number of visitors to the PRWs, compared with other websites.ResultsOverall, the portion of physicians able to be identified on the PRWs ranged from 42.4% (410/966) on OkDoc to 87.3% (843/966) on DocApp. Of the identifiable physicians, only a few of the selected physicians had been rated quantitatively (4.5% [38/843] on DocApp to 49.8% [273/548] on Google) or received narrative comments (4.5% [38/843] on DocApp to 31.2% [171/548] on Google) at least once. Rated physicians also had, on average, a low number of quantitative ratings (1.47 ratings on OkDoc to 3.74 rating on Google) and narrative comments (1.23 comment on OkDoc to 3.03 comments on Google). All 3 websites allowing ratings used the same rating scale (1-5 stars) and had a very positive average rating: DocApp (4.71), Medicosearch (4.69), and Google (4.41). There were significant differences among the PRWs (with the majority of ratings being posted on Google in past 2 years) and regions (with physicians in Zurich more likely to have been rated and have more ratings on average). Only Google (position 1) and Medicosearch (position 8358) are placed among the top 10,000 visited websites in Switzerland.ConclusionsIt appears that this is the first time Google has been included in a study examining physician ratings internationally and it is noticeable how Google has had substantially more ratings than the 3 dedicated PRWs in Switzerland over the past 2 and a half years. Overall, this study indicates that Swiss PRWs are not yet a reliable source of unbiased information regarding patient experiences and satisfaction with Swiss physicians; many selected physicians were unable to be identified, only a few physicians had been rated, and the ratings posted were overwhelmingly positive.
| S-EPMC6688440 | biostudies-literature