Dataset Information

Anchoring vignettes in the Health and Retirement Study: how do medical professionals and disability recipients characterize the severity of work limitations?

ABSTRACT:

Purpose

Recent studies report systematic differences in how individuals categorize the severity of identical health and work limitation vignettes. We investigate how health professionals and disability recipients characterize the severity of work limitations and whether their reporting patterns are robust to demographic, education, and health characteristics. We use the results to illustrate the potential impact of reporting heterogeneity on the distribution of work disability estimated from self-reported categorical health and disability data.

Method

Nationally representative data on anchoring disability vignettes from the 2004 Health and Retirement Study (HRS) are used to investigate how respondents with an occupation background in health and Social Security disability beneficiaries categorize work limitation vignettes. Using pain, cardiovascular health, and depression vignettes, we estimate generalized ordered probit models (N = 2,660 individuals or 39,681 person-vignette observations) that allow the severity thresholds to vary by respondent characteristics.

Results

We find that health professionals (excluding nurses) and disability recipients tend to classify identical work limitations as more severe compared to non-health professional non-disabled respondents. For disability recipients, the differences are most pronounced and particularly visible in the tails of the work limitations distribution. For health professionals, we observe smaller differences, affecting primarily the classification of mildly and moderately severe work limitations. The patterns for health professionals (excluding nurses) are robust to demographics, education, and health conditions. The greater likelihood of viewing the vignette person as more severely work limited observed among disability recipients is mostly explained by the fact that these respondents also tend to be in poorer health which itself predicts a more inclusive scale.

Conclusions

Knowledge of reporting scales from health professionals and disabled individuals can benefit researchers in a broad range of applications in health and disability research. They may be useful as reference scales to evaluate disability survey data. Such knowledge may be beneficial when studying disability programs. Given the increasing availability of anchoring vignette data in surveys, this is a promising area for future evaluation research.

SUBMITTER: Heiland F

PROVIDER: S-EPMC4428751 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundConfigural, metric, and scalar measurement invariance have been indicators of bias-free statistical cross-group comparisons, although they are difficult to verify in the data. Low comparability of translated questionnaires or the different understanding of response formats by respondents might lead to rejection of measurement invariance and point to comparability bias in multi-language surveys. Anchoring vignettes have been proposed as a method to control for the different understanding of response categories by respondents (the latter is referred to as differential item functioning related to response categories or rating scales: RC-DIF). We evaluate the question whether the cross-cultural comparability of data can be assured by means of anchoring vignettes or by considering socio-demographic heterogeneity as an alternative approach.MethodsWe used the Health System Responsiveness (HSR) questionnaire and collected survey data in English (n = 183) and Arabic (n = 121) in a random sample of refugees in the third largest German federal state. We conducted multiple-group Confirmatory Factor Analyses (MGCFA) to analyse measurement invariance and compared the results when 1) using rescaled data on the basis of anchoring vignettes (non-parametric approach), 2) including information on RC-DIF from the analyses with anchoring vignettes as covariates (parametric approach) and 3) including socio-demographic covariates.ResultsFor the HSR, every level of measurement invariance between the Arabic and English languages was rejected. Implementing rescaling or modelling on the basis of anchoring vignettes provided superior results over the initial MGCFA analysis, since configural, metric and - for ordered categorical analyses-scalar invariance could not be rejected. A consideration of socio-demographic variables did not show such an improvement.ConclusionsSurveys may consider anchoring vignettes as a method to assess cross-cultural comparability of data, whereas socio-demographic variables cannot be used to improve data comparability as a standalone method. More research on the efficient implementation of anchoring vignettes and further development of methods to incorporate them when modelling measurement invariance is needed.

Project description:ObjectiveInsurers frequently commission medical experts to estimate the degree of the remaining work capacity (RWC) in claimants for disability benefits. The social functioning scale Mini-ICF-APP allows for a rating of activity and participation limitations in thirteen capacity domains, considered as particularly relevant for work ability. The current study sought to evaluate the role of the Mini-ICF-APP ratings in psychiatric work disability evaluations, by examining how the capacity limitation ratings varied with the claimants' primary psychiatric diagnoses and how the ratings were related to RWC estimates.MethodsMedical experts estimated the RWC of 946 claimants with mental disorders and rated their activity and participation limitations using the Mini-ICF-APP, with higher ratings reflecting more severe limitations. The ratings were compared between claimants with different psychiatric diagnoses by analyses of variance. The mean Mini-ICF-APP rating across all capacity domains as well as all capacity-specific ratings were entered in simple or multiple regression models to predict the RWC in an alternative job.ResultsThe Mini-ICF-APP capacity limitation ratings in all domains but mobility were higher for claimants with personality and behavior disorders as compared to those with mood disorders or with neurotic, stress-related and somatoform disorders. The largest differences were observed in social capacities (e.g. group integration: F 2, 847 = 78.300, P < 0.001). In claimants with depression, all ratings increased with the severity of the diagnosis (all Fs 2, 203 > 16.393, all Ps < 0.001). In the overall sample, the mean Mini-ICF-APP rating showed a strong negative correlation with the estimated RWC (r = -.720, P < 0.001). Adding the capacity-specific ratings to the prediction model improved this prediction only marginally.DiscussionThe Mini-ICF-APP allows for documenting claimants' activity and participation limitations, which is likely to increase the transparency of medical experts' RWC estimates and enables them to check the plausibility of such estimates. However, our study showed that despite the strong association between RWC and Mini-ICF-APP ratings, half of the RWC variance was unrelated to the capacity limitations documented in the Mini-ICF-APP.

Project description:BACKGROUND:With India preparing for the next decennial Census in 2021, we compared the disability estimates and data collection methodology between the Census 2011 and the most recent population-level survey for India and its states, to highlight the issues to be addressed to improve robustness of the disability estimates in the upcoming Census. METHODS:Data from the Census 2011 and from two complementary nationally representative household surveys that covered all Indian states with the same methodology and survey instruments-the District-Level Household Survey-4 (DLHS-4, 2012-2013) and the Annual Health Surveys (AHS three rounds, 2010-11, 2011-12 and 2012-13) were used. Data from DLHS-4 and AHS 2012-13 round were pooled to generate estimates for the year 2012-13. Data collection methodology between the sources was compared, including the review of definitions of each type of disability. The overall, mental, visual, hearing, speech, and movement disability rate (DR) per 100,000 population were compared between the sources for India and for each state, and the percent difference in the respective rates was calculated. We explored the reliability of these estimates comparing yearly data from the AHS for three successive rounds. RESULTS:Survey data were collected through proxy reporting, however, it is not entirely clear whether the data were proxy- or self-reported or a mix of both in the Census. The overall DR was 25.1% higher in the Census (2,242; 95% CI 2,241-2,243) than the survey (1,791; 95% CI 1,786-1,797) per 100,000 population, with the state-level difference ranging from -64% in Tamil Nadu to 107% in Sikkim state. Despite both sources using nearly similar definitions for overall disability and disability by type, the difference in DR was 125.5%, 54.2%, -25.7%, -19.7%, and 21.9% for hearing, speech, mental, movement, and visual DR, respectively. At the state-level, the difference in disability-specific estimates ranged from -84% to 450%. The extent of variations in the disability-specific estimates in AHS successive rounds ranged from -25% to 929% at the state-level. CONCLUSIONS:There is momentum globally towards building disability measurement that is consistent with the data required for monitoring of the Sustainable Development Goals to ensure robust estimation of disability. The current estimates from the Census and surveys seem much lower than would be expected at the population level. We make recommendations that India needs to take serious note of in order to improve the validity and reliability of India's disability estimates.