Dataset Information

Psychometric evaluation of a decision quality instrument for medication decisions for treatment of depression symptoms.

ABSTRACT:

Background

A high quality treatment decision means patients are informed and receive treatment that matches their goals. This research examined the reliability and validity of the Depression Decision Quality Instrument (DQI), a survey to measure the extent to which patients are informed and received preferred treatment for depression.

Methods

Participants were aged 18 and older from 17 US cities who discussed medication or counseling with a physician in the past year, and physicians who treated patients with depression who practiced in the same cities. Participants were mailed a survey that included the Depression-DQI, a tool with 10 knowledge and 7 goal and concern items. Patients were randomly assigned to either receive a patient decision aid (DA) on treatment of depression or no DA. A matching score was created by comparing the patient's preferred treatment to their self-reported treatment received. Concordant scores were considered matched, discordant were not. We examined the reliability and known group validity of the Depression-DQI.

Results

Most patients 405/504 (80%) responded, 79% (320/405) returned the retest survey, and 60% (114/187) of physicians returned the survey. Patients' knowledge scores on the 10-item scale ranged from 14.6 to 100% with no evidence of floor or ceiling effects. Retest reliability for knowledge was moderate and for goals and concerns ranged from moderate to good. Mean knowledge scores differentiated between patients and physicians (M = 63 [SD = 15] vs. M = 81 [SD = 11], p < 0.001), and between patients who did and didn't receive a DA (M = 64 [SD = 16] vs. M = 61 [SD = 14], p = 0.041). 60.5% of participants received treatment that matched their preference. Based on the multivariate logistic regression, 'avoiding taking anti-depressants' was the only goal that was predictive of taking mediation (OR = 0.73 [0.66, 0.80], p < 0.01). Shared Decision Making Process scores were similar for those who matched their preference and those who didn't (M = 2.18 [SD = 0.97] vs. M = 2.06 [SD = 1.07]; t(320) = - 1.06, p = 0.29). Those who matched had lower regret scores (matched M = 1.72 [SD = 0.74] vs. unmatched M = 2.32 [SD = 0.8]; t(301) = - 6.6, p < .001).

Conclusions

The Depression DQI demonstrated modest reliability and validity. More work is needed to establish validity of the method to determine concordance.

Trial registration

NCT01152307.

SUBMITTER: Brodney S

PROVIDER: S-EPMC8394109 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundWomen diagnosed with early stage (I or II) breast cancer face a highly challenging decision - whether or not to undergo adjuvant chemotherapy. We developed a decision quality instrument for chemotherapy for early stage breast cancer and sought to evaluate its performance.MethodsCross-sectional, mailed survey of recent breast cancer survivors, providers, and healthy controls and a retest survey of survivors. The decision quality instrument includes questions on knowledge and personal goals. It results in a knowledge score and concordance score, which reflects the percentage of patients who received treatments that match their goals. Hypotheses related to acceptability, feasibility, validity, and reliability of the survey instrument were examined.ResultsResponses were received from 352 patients, 89 providers and 35 healthy controls. The decision quality instrument was feasible to implement with few missing data. The knowledge scores had good retest reliability (intraclass correlation coefficient (ICC) =0.75). Knowledge scores discriminated between providers and patients (mean difference 31.1%, 95% CI 26.9, 35.3) and between patients and healthy controls (mean difference 11.2, 95% CI 5.4, 17.1). Most providers reported that the knowledge items covered essential content. Two of the five goal items had a ceiling effect, and one goal had low content validity. The goal items had moderate retest reliability (ICC's 0.57 to 0.78). In the multivariable model of treatment, none of the patient goals was associated with receipt of chemotherapy. Age and hormone receptor status were the only variables independently associated with chemotherapy. Most patients (77.6%) had treatment concordant with that predicted by the model. Patients who had concordant treatment had similar levels of confidence and regret as those who did not.ConclusionsThe Decision Quality Instrument is a reliable and valid measure of patient knowledge about chemotherapy, but its ability to measure concordance with patient goals is limited. In this sample, patient goals were not associated with treatment, and most patients reported they were not asked their preference, suggesting that goals were not adequately considered in decision making.

Project description:Nighttime symptoms can negatively impact the quality of life of patients with chronic obstructive pulmonary disease (COPD). The Nighttime Symptoms of COPD Instrument (NiSCI) was designed to measure the occurrence and severity of nighttime symptoms in patients with COPD, the impact of symptoms on nighttime awakenings, and rescue medication use. The objective of this study was to explore item reduction, inform scoring recommendations, and evaluate the psychometric properties of the NiSCI.COPD patients participating in a Phase III clinical trial completed the NiSCI daily. Item analyses were conducted using weekly mean and single day scores. Descriptive statistics (including percentage of respondents at floor/ceiling and inter-item correlations), factor analyses, and Rasch model analyses were conducted to examine item performance and scoring. Test-retest reliability was assessed for the final instrument using the intraclass correlation coefficient (ICC). Correlations with assessments conducted during study visits were used to evaluate convergent and known-groups validity.Data from 1,663 COPD patients aged 40-93 years were analyzed. Item analyses supported the generation of four scores. A one-factor structure was confirmed with factor analysis and Rasch analysis for the symptom severity score. Test-retest reliability was confirmed for the six-item symptom severity (ICC, 0.85), number of nighttime awakenings (ICC, 0.82), and rescue medication (ICC, 0.68) scores. Convergent validity was supported by significant correlations between the NiSCI, St George's Respiratory Questionnaire, and Exacerbations of Chronic Obstructive Pulmonary Disease Tool-Respiratory Symptoms scores.The results suggest that the NiSCI can be used to determine the severity of nighttime COPD symptoms, the number of nighttime awakenings due to COPD symptoms, and the nighttime use of rescue medication. The NiSCI is a reliable and valid instrument to evaluate these concepts in COPD patients in clinical trials and clinical practice. Scoring recommendations and steps for further research are discussed.

Project description:BackgroundEarly morning respiratory symptoms impact quality of life and are often the most troublesome for patients with COPD. Reduction in symptoms and their impact are important treatment outcomes for COPD. The Early Morning Symptoms of COPD Instrument (EMSCI) is a daily diary designed to collect patients' report of the occurrence, severity, and impact of morning COPD symptoms.MethodsTo assess the psychometric properties of the EMSCI, a split-half sample of data from a COPD clinical trial where participants completed the EMSCI daily was used for conducting descriptive statistics, factor analyses, and Rasch model analyses to examine item performance and inform scoring. Once the final scoring algorithm was determined, data from the second split-half sample were used to examine the properties of the EMSCI. Test-retest reliability was assessed using intraclass correlation coefficient (ICC). Correlations with other study assessments were used to evaluate convergent and known-groups validity.ResultsData from 1,663 patients with COPD aged 40-93 years were analyzed. Factor analysis and Rasch analysis confirmed a one-factor structure for the 6 individual symptom items. Item analyses supported the generation of 4 scores. All scores demonstrated good test-retest reliability: 6-item symptom severity (ICC, 0.84); overall morning symptom severity (ICC, 0.84); activity limitation (ICC, 0.85); and rescue medication (ICC, 0.62). Significant correlations between EMSCI scores, St George's Respiratory Questionnaire scores, and EXAcerbations of Chronic pulmonary disease Tool (EXACT)-Respiratory Symptoms scores supported the tool's convergent validity. Significant differences (p<0.0001) in all EMSCI domain scores were found between known-groups based on median split St George's Respiratory Questionnaire and EXACT-Respiratory Symptoms scores.ConclusionThe EMSCI consists of 4 scores: 6-item symptom severity, overall symptom severity, activity limitation, and rescue medication. The EMSCI is a reliable and valid instrument for evaluating patients' experience of early morning COPD symptoms.

Project description:BackgroundGuidelines for colorectal cancer screening recommend that patients be informed about options and be able to select preferred method of screening; however, there are no existing measures available to assess whether this happens.MethodsColorectal Cancer Screening Decision Quality Instrument (CRC-DQI) includes knowledge items and patients' goals and concerns. Items were generated through literature review and qualitative work with patients and providers. Hypotheses relating to the acceptability, feasibility, discriminant validity and retest reliability of the survey were examined using data from three studies: (1) 2X2 randomized study of participants recruited online, (2) cross-sectional sample of patients recruited in community health clinics, and (3) cross-sectional sample of providers recruited from American Medical Association Master file.Results338 participants were recruited online, 94 participants were recruited from community health centers, and 115 physicians were recruited. The CRC-DQI was feasible and acceptable with low missing data and high response rates for both online and paper-based administrations. The knowledge score was able to discriminate between those who had seen a decision aid or not (84% vs. 64%, p < 0.001) and between providers, online patients and clinic patients (89% vs. 74% vs. 41%, p < 0.001 for all comparisons). The knowledge score and most of the goals had adequate retest reliability. About half of the participants received a test that matched their goals (47% and 51% in online and clinic samples respectively). Many respondents who had never been screened had goals that indicated a preference for colonoscopy. A minority of respondents in the online (21%) and in clinic (2%) samples were both well informed and received a test that matched their goals.ConclusionsThe CRC-DQI demonstrated good psychometric properties in diverse samples, and across different modes of administration. Few respondents made high quality decisions about colon cancer screening.