Dataset Information

Does effectiveness in performance appraisal improve with rater training?

ABSTRACT: Performance appraisal is a complex process by which an organization can determine the extent to which employees are performing their work effectively. However, this appraisal may not be accurate if there is no reduction in the impact of problems caused by possibly subjective rater judgements. The main objective of this work is to check the effectiveness-separately and jointly-of the following four training programmes in the extant literature aimed at improving the accuracy of performance assessment: 1) Performance Dimension Training, 2) Frame-of-Reference, 3) Rater Error Training, and 4) Behavioural Observation Training. Based on these training strategies, three programmes were designed and applied separately. A fourth programme was a combination of the other three. We analyzed two studies using different samples (85 students and 42 employees) for the existence of differences in the levels of knowledge of performance and its dimensions, rater errors, observational accuracy, and accuracy of task and citizenship performance appraisal, according to the type of training raters receive. First, the main results show that training based on performance dimensions and the creation of a common framework, in addition to the training that includes the four programmes (Training_4_programmes), increases the level of knowledge of performance and its dimensions. Second, groups that receive training in rater error score higher in knowledge of biases than the other groups, whether or not they have received training. Third, participants' observational accuracy improves with each new moment measure (post-training and follow-up), though not because of the type of training received. Fourth, participants who receive training through the programme that combine the other four gave a task performance appraisal that was closer to the one undertaken by the judges-experts than the other groups. And finally, students' citizenship performance appraisal does not vary according to type of training or to different moment measures, whereas the group of employees who received all four types of training gave a more accurate citizenship performance assessment.

SUBMITTER: Rosales Sanchez C

PROVIDER: S-EPMC6752840 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Does effectiveness in performance appraisal improve with rater training?

Rosales Sánchez Christian C Díaz-Cabrera Dolores D Hernández-Fernaud Estefanía E

PloS one 20190919 9

Performance appraisal is a complex process by which an organization can determine the extent to which employees are performing their work effectively. However, this appraisal may not be accurate if there is no reduction in the impact of problems caused by possibly subjective rater judgements. The main objective of this work is to check the effectiveness-separately and jointly-of the following four training programmes in the extant literature aimed at improving the accuracy of performance assessm ...[more]

PMID: 31536562

Similar Datasets

Project description:BackgroundReduced muscle strength is an independent risk factor for falls and related to postural instability in individuals with Parkinson's disease. The ability of resistance training to improve postural control still remains unclear.ObjectiveTo compare resistance training with balance training to improve postural control in people with Parkinson's disease.Methods40 patients with idiopathic Parkinson's disease (Hoehn&Yahr: 2.5-3.0) were randomly assigned into resistance or balance training (2x/week for 7 weeks). Assessments were performed at baseline, 8- and 12-weeks follow-up: primary outcome: Fullerton Advanced Balance (FAB) scale; secondary outcomes: center of mass analysis during surface perturbations, Timed-up-and-go-test, Unified Parkinson's Disease Rating Scale, Clinical Global Impression, gait analysis, maximal isometric leg strength, PDQ-39, Beck Depression Inventory. Clinical tests were videotaped and analysed by a second rater, blind to group allocation and assessment time.Results32 participants (resistance training: n = 17, balance training: n = 15; 8 drop-outs) were analyzed at 8-weeks follow-up. No significant difference was found in the FAB scale when comparing the effects of the two training types (p = 0.14; effect size (Cohen's d) = -0.59). Participants from the resistance training group, but not from the balance training group significantly improved on the FAB scale (resistance training: +2.4 points, Cohen's d = -0.46; balance training: +0.3 points, Cohen's d = -0.08). Within the resistance training group, improvements of the FAB scale were significantly correlated with improvements of rate of force development and stride time variability. No significant differences were found in the secondary outcome measures when comparing the training effects of both training types.ConclusionsThe difference between resistance and balance training to improve postural control in people with Parkinson's disease was small and not significant with this sample size. There was weak evidence that freely coordinated resistance training might be more effective than balance training. Our results indicate a relationship between the enhancement of rate of force development and the improvement of postural control.Trial registrationClinicalTrials.gov ID: NCT02253563.

Project description:BackgroundPoor inter-rater reliability in chest radiograph interpretation has been reported in the context of acute respiratory distress syndrome (ARDS), although not for the Berlin definition of ARDS. We sought to examine the effect of training material on the accuracy and consistency of intensivists' chest radiograph interpretations for ARDS diagnosis.MethodsWe conducted a rater agreement study in which 286 intensivists (residents 41.3%, junior attending physicians 35.3%, and senior attending physician 23.4%) independently reviewed the same 12 chest radiographs developed by the ARDS Definition Task Force ("the panel") before and after training. Radiographic diagnoses by the panel were classified into the consistent (n = 4), equivocal (n = 4), and inconsistent (n = 4) categories and were used as a reference. The 1.5-hour training course attended by all 286 intensivists included introduction of the diagnostic rationale, and a subsequent in-depth discussion to reach consensus for all 12 radiographs.ResultsOverall diagnostic accuracy, which was defined as the percentage of chest radiographs that were interpreted correctly, improved but remained poor after training (42.0 ± 14.8% before training vs. 55.3 ± 23.4% after training, p < 0.001). Diagnostic sensitivity and specificity improved after training for all diagnostic categories (p < 0.001), with the exception of specificity for the equivocal category (p = 0.883). Diagnostic accuracy was higher for the consistent category than for the inconsistent and equivocal categories (p < 0.001). Comparisons of pre-training and post-training results revealed that inter-rater agreement was poor and did not improve after training, as assessed by overall agreement (0.450 ± 0.406 vs. 0.461 ± 0.575, p = 0.792), Fleiss's kappa (0.133 ± 0.575 vs. 0.178 ± 0.710, p = 0.405), and intraclass correlation coefficient (ICC; 0.219 vs. 0.276, p = 0.470).ConclusionsThe radiographic diagnostic accuracy and inter-rater agreement were poor when the Berlin radiographic definition was used, and were not significantly improved by the training set of chest radiographs developed by the ARDS Definition Task Force.Trial registrationThe study was registered at ClinicalTrials.gov (registration number NCT01704066 ) on 6 October 2012.

Project description:BACKGROUND: Organisational culture is an anthropological metaphor used to inform research and consultancy and to explain organisational environments. In recent years, increasing emphasis has been placed on the need to change organisational culture in order to improve healthcare performance. However, the precise function of organisational culture in healthcare policy often remains underspecified and the desirability and feasibility of strategies to be adopted have been called into question. The objective of this review was to determine the effectiveness of strategies to change organisational culture in order to improve healthcare performance. METHODS: We searched the following electronic databases: The Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, CINAHL, Sociological Abstracts, Web of Knowledge, PsycINFO, Business and Management, EThOS, Index to Theses, Intute, HMIC, SIGLE, and Scopus until October 2009. The Database of Abstracts of Reviews of Effectiveness (DARE) was searched for related reviews. We also searched the reference lists of all papers and relevant reviews identified, and we contacted experts in the field for advice on further potential studies. We considered randomised controlled trials (RCTs) or well designed quasi-experimental studies (controlled clinical trials (CCTs), controlled before and after studies (CBAs), and interrupted time series (ITS) analyses). Studies could be set in any type of healthcare organisation in which strategies to change organisational culture in order to improve healthcare performance were applied. Our main outcomes were objective measures of professional performance and patient outcome. RESULTS: The search strategy yielded 4,239 records. After the full text assessment, two CBA studies were included in the review. They both assessed the impact of interventions aimed at changing organisational culture, but one evaluated the impact on work-related and personal outcomes while the other measured clinical outcomes. Both were at high risk of bias. Both reported positive results. CONCLUSIONS: Current available evidence does not identify any effective, generalisable strategies to change organisational culture. Healthcare organisations considering implementing interventions aimed at changing culture should seriously consider conducting an evaluation (using a robust design, e.g., ITS) to strengthen the evidence about this topic.

Dataset Information

Does effectiveness in performance appraisal improve with rater training?

Publications

Does effectiveness in performance appraisal improve with rater training?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets