Dataset Information

Critical examination of current response shift methods and proposal for advancing new methods.

ABSTRACT:

Purpose

This work is part of an international, interdisciplinary initiative to synthesize research on response shift in results of patient-reported outcome measures. The objective is to critically examine current response shift methods. We additionally propose advancing new methods that address the limitations of extant methods.

Methods

Based on literature reviews, this critical examination comprises design-based, qualitative, individualized, and preference-based methods, latent variable models, and other statistical methods. We critically appraised their definition, operationalization, the type of response shift they can detect, whether they can adjust for and explain response shift, their assumptions, and alternative explanations. Overall limitations requiring new methods were identified.

Results

We examined 11 methods that aim to operationalize response shift, by assessing change in the meaning of one's self-evaluation. Six of these methods distinguish between change in observed measurements (observed change) and change in the construct that was intended to be measured (target change). The methods use either (sub)group-based or individual-level analysis, or a combination. All methods have underlying assumptions to be met and alternative explanations for the inferred response shift effects. We highlighted the need to address the interpretation of the results as response shift and proposed advancing new methods handling individual variation in change over time and multiple time points.

Conclusion

No single response shift method is optimal; each method has strengths and limitations. Additionally, extra steps need to be taken to correctly interpret the results. Advancing new methods and conducting computer simulation studies that compare methods are recommended to move response shift research forward.

SUBMITTER: Sebille V

PROVIDER: S-EPMC8602164 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Introduction: Specialist medical assessments fulfil the task of ensuring that physicians have the clinical competence to independently represent their field and provide the best possible care to patients, taking into account the current state of knowledge. To date, there are no comprehensive reports on the status of specialist assessments in the German-speaking countries (DACH). For that reason, the assessment methods used in the DACH region are compiled and critically evaluated in this article, and recommendations for further development are described. Methods: The websites of the following institutions were searched for information regarding testing methods used and the organisation of specialist examinations: Homepage of the Swiss Institute for Medical Continuing Education (SIWF), Homepage of the Academy of Physicians (Austria) and Homepage of the German Federal Medical Association (BAEK). Further links were considered and the results were presented in tabular form. The assessment methods used in the specialist assessments are critically examined with regard to established quality criteria and recommendations for the further development of the specialist assessments are derived from these. Results: The following assessment methods are already used in Switzerland and Austria: written examinations with multiple choice and short answer questions, structured oral examinations, the Script Concordance Test (SCT) and the Objective Structured Clinical Examination (OSCE). In some cases, these assessment methods are combined (triangulation). In Germany, on the other hand, the oral examination has so far been conducted in an unstructured manner in the form of a 'collegial content discussion'. In order to test knowledge, practical and communicative competences equally, it is recommended to implement a triangulation of methods and follow the further recommendations described in this article. Conclusion: While there are already accepted approaches for quality-assured and competence-based specialist assessments in Switzerland and Austria at present, there is still a long way to go in Germany. Following the recommendations presented in this article, a contribution could be made to improving the specialist assessments in the DACH region according to the specialist assessments objectives.