Dataset Information

The Effects of Vignette Scoring on Reliability and Validity of Self-Reports.

ABSTRACT: The research presented in this article combines mathematical derivations and empirical results to investigate effects of the nonparametric anchoring vignette approach proposed by King, Murray, Salomon, and Tandon on the reliability and validity of rating data. The anchoring vignette approach aims to correct rating data for response styles to improve comparability across individuals and groups. Vignettes are used to adjust self-assessment responses on the respondent level but entail significant assumptions: They are supposed to be invariant across respondents, and the responses to vignette prompts are supposed to be without error and strictly ordered. This article shows that these assumptions are not always met and that the anchoring vignette approach leads to higher Cronbach's alpha values and increased correlations among adjusted variables regardless of whether the assumptions of the approach are met or violated. Results suggest that the underlying assumptions and effects of the anchoring vignette approach should be carefully examined as the increased correlations and reliability estimates can be observed even for response variables that are independent random draws and uncorrelated with any other variable.

SUBMITTER: von Davier M

PROVIDER: S-EPMC5978608 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The Effects of Vignette Scoring on Reliability and Validity of Self-Reports.

von Davier Matthias M Shin Hyo-Jeong HJ Khorramdel Lale L Stankov Lazar L

Applied psychological measurement 20170927 4

The research presented in this article combines mathematical derivations and empirical results to investigate effects of the nonparametric anchoring vignette approach proposed by King, Murray, Salomon, and Tandon on the reliability and validity of rating data. The anchoring vignette approach aims to correct rating data for response styles to improve comparability across individuals and groups. Vignettes are used to adjust self-assessment responses on the respondent level but entail significant a ...[more]

PMID: 29881126

Similar Datasets

Project description:Background: Functional or quality of life questionnaires are important tools in clinical investigations. The Lysholm Knee Scoring Scale and Tegner Activity Scale are knee-specific questionnaires that are widely used to assess knee function. Purpose: To translate both questionnaires into Thai and to assess the validity and reliability of the Thai versions of the Lysholm and adjusted Tegner scales. Study Design: Cohort study (diagnosis); Level of evidence, 3. Methods: The Lysholm and Tegner scales were translated into Thai by using the forward-backward translation protocol. Because cultural modifications were made to the sports used to measure activity on the Tegner scale, the authors of this study refer to the Thai version as the “Thai adjusted Tegner scale.” The reliability and validity of the translated scales were evaluated by obtaining the responses of 60 consecutive patients (mean age, 40.5 years; 34 male, 26 female); the patients also completed the Thai version of the International Knee Documentation Committee Subjective Knee Form (IKDC-SKF). Criterion validity was tested by correlating the scores from both translated questionnaires with those from the Thai IKDC-SKF, while reliability was assessed by measuring test-retest reliability and internal consistency. Results: The Thai Lysholm scale showed a strong correlation with the Thai IKDC-SKF (r = 0.89), while the Thai adjusted Tegner scale showed a moderate correlation with the Thai IKDC-SKF (r = 0.60). The intrarater and test-retest reliability measures were excellent for the Thai Lysholm (intraclass correlation coefficient [ICC], 0.94 and 0.98, respectively) and moderate to good for the Thai adjusted Tegner (ICC, 0.73 and 0.86, respectively). The internal consistency for the Thai Lysholm was acceptable at all the time points (Cronbach alpha, 0.71-0.73). Conclusion: The Thai Lysholm and Thai adjusted Tegner scales adequately retained the characteristics of the original versions. They can be considered reliable instruments for Thai patients with knee-related problems.

Project description:Background:With the rapid increase in ageing population, China is confronted with the daunting challenge of a growing number of patients with neurocognitive disorders (NCDs). This trend makes the maintenance of self-health and early intervention essential, highlighting the need for a tool that assesses self-efficacy of older adults in maintaining brain health or cognitive function. Aim:This study aimed to design the Brain Health Self-Efficacy Scale (BHSES) to measure elderly individuals' attitudes to NCDs, motivations and future plans for controlling risks. The psychometric properties of BHSES have been validated. Methods:Based on the current literature and relevant models, a 19-item scale was created during the first stage. A total of 660 older adults in the Yinhang community of Shanghai were included. The statistical approaches of item analysis, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), criterion-related validity and reliability test were used to evaluate the quality of BHSES. In addition, the Geriatric Depression Scale (GDS) and the Self-Rating Anxiety Scale (SAS) were used as criteria to test the criterion-related validity. Results:To test item differentiation, the study adopted item analysis and excluded item 8. Selecting a random half of the sample for EFA, the BHSES was refined to 16 items, which were categorised into the following three dimensions: 'memory belief efficacy', 'self-care efficacy' and 'future planning efficacy'. These were highly consistent with the hypothesis model. Its cumulative variance contribution rate reached 61.14%, with factor loads of all items above 0.5. The three-factor model was confirmed by the remaining data through CFA. All fit indices reached the acceptable level (?2=3.045, Goodness of Fit Index=0.898, adjusted Goodness of Fit Index=0.863, Comparative Fit Index=0.916, Incremental Fit Index=0.917, Tucker-Lewis Index=0.900, root mean square error of approximation=0.079 and root mean residual=0.068). The GDS and SAS scores revealed significant correlations with the BHSES score, indicating a high criterion-related validity. The overall Cronbach's ? coefficient was 0.793, with the ? coefficients' distribution of subdimensions ranging from 0.748 to 0.883. Conclusions:The 16-item, self-compiled BHSES is a reliable and valid measurement. It could help identify older adults with potential risks for developing NCDs or with high suspicion of cognitive impairment onset in recent periods and also offer insight into tracking brain health self-efficacy in association with cognition status.

Dataset Information

The Effects of Vignette Scoring on Reliability and Validity of Self-Reports.

Publications

The Effects of Vignette Scoring on Reliability and Validity of Self-Reports.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets