Dataset Information

Assessing the equivalence of Web-based and paper-and-pencil questionnaires using differential item and test functioning (DIF and DTF) analysis: a case of the Four-Dimensional Symptom Questionnaire (4DSQ).

ABSTRACT: PURPOSE:Many paper-and-pencil (P&P) questionnaires have been migrated to electronic platforms. Differential item and test functioning (DIF and DTF) analysis constitutes a superior research design to assess measurement equivalence across modes of administration. The purpose of this study was to demonstrate an item response theory (IRT)-based DIF and DTF analysis to assess the measurement equivalence of a Web-based version and the original P&P format of the Four-Dimensional Symptom Questionnaire (4DSQ), measuring distress, depression, anxiety, and somatization. METHODS:The P&P group (n?=?2031) and the Web group (n?=?958) consisted of primary care psychology clients. Unidimensionality and local independence of the 4DSQ scales were examined using IRT and Yen's Q3. Bifactor modeling was used to assess the scales' essential unidimensionality. Measurement equivalence was assessed using IRT-based DIF analysis using a 3-stage approach: linking on the latent mean and variance, selection of anchor items, and DIF testing using the Wald test. DTF was evaluated by comparing expected scale scores as a function of the latent trait. RESULTS:The 4DSQ scales proved to be essentially unidimensional in both modalities. Five items, belonging to the distress and somatization scales, displayed small amounts of DIF. DTF analysis revealed that the impact of DIF on the scale level was negligible. CONCLUSIONS:IRT-based DIF and DTF analysis is demonstrated as a way to assess the equivalence of Web-based and P&P questionnaire modalities. Data obtained with the Web-based 4DSQ are equivalent to data obtained with the P&P version.

SUBMITTER: Terluin B

PROVIDER: S-EPMC5891556 | biostudies-literature | 2018 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing the equivalence of Web-based and paper-and-pencil questionnaires using differential item and test functioning (DIF and DTF) analysis: a case of the Four-Dimensional Symptom Questionnaire (4DSQ).

Terluin Berend B Brouwers Evelien P M EPM Marchand Miquelle A G MAG de Vet Henrica C W HCW

Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation 20180221 5

<h4>Purpose</h4>Many paper-and-pencil (P&P) questionnaires have been migrated to electronic platforms. Differential item and test functioning (DIF and DTF) analysis constitutes a superior research design to assess measurement equivalence across modes of administration. The purpose of this study was to demonstrate an item response theory (IRT)-based DIF and DTF analysis to assess the measurement equivalence of a Web-based version and the original P&P format of the Four-Dimensional Symptom Questio ...[more]

PMID: 29468387

Similar Datasets

Project description:BackgroundEfficient screening questionnaires are useful in general practice. Computerized adaptive testing (CAT) is a method to improve the efficiency of questionnaires, as only the items that are particularly informative for a certain responder are dynamically selected.ObjectiveThe objective of this study was to test whether CAT could improve the efficiency of the Four-Dimensional Symptom Questionnaire (4DSQ), a frequently used self-report questionnaire designed to assess common psychosocial problems in general practice.MethodsA simulation study was conducted using a sample of Dutch patients visiting a general practitioner (GP) with psychological problems (n=379). Responders completed a paper-and-pencil version of the 50-item 4DSQ and a psychometric evaluation was performed to check if the data agreed with item response theory (IRT) assumptions. Next, a CAT simulation was performed for each of the four 4DSQ scales (distress, depression, anxiety, and somatization), based on the given responses as if they had been collected through CAT. The following two stopping rules were applied for the administration of items: (1) stop if measurement precision is below a predefined level, or (2) stop if more than half of the items of the subscale are administered.ResultsIn general, the items of each of the four scales agreed with IRT assumptions. Application of the first stopping rule reduced the length of the questionnaire by 38% (from 50 to 31 items on average). When the second stopping rule was also applied, the total number of items could be reduced by 56% (from 50 to 22 items on average).ConclusionsCAT seems useful for improving the efficiency of the 4DSQ by 56% without losing a considerable amount of measurement precision. The CAT version of the 4DSQ may be useful as part of an online assessment to investigate the severity of mental health problems of patients visiting a GP. This simulation study is the first step needed for the development a CAT version of the 4DSQ. A CAT version of the 4DSQ could be of high value for Dutch GPs since increasing numbers of patients with mental health problems are visiting the general practice. In further research, the results of a real-time CAT should be compared with the results of the administration of the full scale.

Project description:BackgroundThe National Institute on Alcohol Abuse and Alcoholism (NIAAA) recommends the paper-based or computerized Alcohol Symptom Checklist to assess alcohol use disorder (AUD) symptoms in routine care when patients report high-risk drinking. However, it is unknown whether Alcohol Symptom Checklist response characteristics differ when it is administered online (eg, remotely via an online electronic health record [EHR] patient portal before an appointment) versus in clinic (eg, on paper after appointment check-in).ObjectiveThis study evaluated the psychometric performance of the Alcohol Symptom Checklist when completed online versus in clinic during routine clinical care.MethodsThis cross-sectional, psychometric study obtained EHR data from the Alcohol Symptom Checklist completed by adult patients from an integrated health system in Washington state. The sample included patients who had a primary care visit in 2021 at 1 of 32 primary care practices, were due for annual behavioral health screening, and reported high-risk drinking on the behavioral health screen (Alcohol Use Disorder Identification Test-Consumption score ≥7). After screening, patients with high-risk drinking were typically asked to complete the Alcohol Symptom Checklist-an 11-item questionnaire on which patients self-report whether they had experienced each of the 11 AUD criteria listed in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) over a past-year timeframe. Patients could complete the Alcohol Symptom Checklist online (eg, on a computer, smartphone, or tablet from any location) or in clinic (eg, on paper as part of the rooming process at clinical appointments). We examined sample and measurement characteristics and conducted differential item functioning analyses using item response theory to examine measurement consistency across these 2 assessment modalities.ResultsAmong 3243 patients meeting eligibility criteria for this secondary analysis (2313/3243, 71% male; 2271/3243, 70% White; and 2014/3243, 62% non-Hispanic), 1640 (51%) completed the Alcohol Symptom Checklist online while 1603 (49%) completed it in clinic. Approximately 46% (752/1640) and 48% (764/1603) reported ≥2 AUD criteria (the threshold for AUD diagnosis) online and in clinic (P=.37), respectively. A small degree of differential item functioning was observed for 4 of 11 items. This differential item functioning produced only minimal impact on total scores used clinically to assess AUD severity, affecting total criteria count by a maximum of 0.13 criteria (on a scale ranging from 0 to 11).ConclusionsCompleting the Alcohol Symptom Checklist online, typically prior to patient check-in, performed similarly to an in-clinic modality typically administered on paper by a medical assistant at the time of the appointment. Findings have implications for using online AUD symptom assessments to streamline workflows, reduce staff burden, reduce stigma, and potentially assess patients who do not receive in-person care. Whether modality of DSM-5 assessment of AUD differentially impacts treatment is unknown.

Project description:BackgroundThe Four-Dimensional Symptom Questionnaire (4DSQ) is a self-report questionnaire that has been developed in primary care to distinguish non-specific general distress from depression, anxiety and somatization. The purpose of this paper is to evaluate its criterion and construct validity.MethodsData from 10 different primary care studies have been used. Criterion validity was assessed by comparing the 4DSQ scores with clinical diagnoses, the GPs' diagnosis of any psychosocial problem for Distress, standardised psychiatric diagnoses for Depression and Anxiety, and GPs' suspicion of somatization for Somatization. ROC analyses and logistic regression analyses were used to examine the associations. Construct validity was evaluated by investigating the inter-correlations between the scales, the factorial structure, the associations with other symptom questionnaires, and the associations with stress, personality and social functioning. The factorial structure of the 4DSQ was assessed through confirmatory factor analysis (CFA). The associations with other questionnaires were assessed with Pearson correlations and regression analyses.ResultsRegarding criterion validity, the Distress scale was associated with any psychosocial diagnosis (area under the ROC curve [AUC] 0.79), the Depression scale was associated with major depression (AUC = 0.83), the Anxiety scale was associated with anxiety disorder (AUC = 0.66), and the Somatization scale was associated with the GPs' suspicion of somatization (AUC = 0.65). Regarding the construct validity, the 4DSQ scales appeared to have considerable inter-correlations (r = 0.35-0.71). However, 30-40% of the variance of each scale was unique for that scale. CFA confirmed the 4-factor structure with a comparative fit index (CFI) of 0.92. The 4DSQ scales correlated with most other questionnaires measuring corresponding constructs. However, the 4DSQ Distress scale appeared to correlate with some other depression scales more than the 4DSQ Depression scale. Measures of stress (i.e. life events, psychosocial problems, and work stress) were mainly associated with Distress, while Distress, in turn, was mainly associated with psychosocial dysfunctioning, including sick leave.ConclusionThe 4DSQ seems to be a valid self-report questionnaire to measure distress, depression, anxiety and somatization in primary care patients. The 4DSQ Distress scale appears to measure the most general, most common, expression of psychological problems.

Dataset Information

Assessing the equivalence of Web-based and paper-and-pencil questionnaires using differential item and test functioning (DIF and DTF) analysis: a case of the Four-Dimensional Symptom Questionnaire (4DSQ).

Publications

Assessing the equivalence of Web-based and paper-and-pencil questionnaires using differential item and test functioning (DIF and DTF) analysis: a case of the Four-Dimensional Symptom Questionnaire (4DSQ).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets