Dataset Information

Screening for Depression in Daily Life: Development and External Validation of a Prediction Model Based on Actigraphy and Experience Sampling Method.

ABSTRACT:

Background

In many countries, depressed individuals often first visit primary care settings for consultation, but a considerable number of clinically depressed patients remain unidentified. Introducing additional screening tools may facilitate the diagnostic process.

Objective

This study aimed to examine whether experience sampling method (ESM)-based measures of depressive affect and behaviors can discriminate depressed from nondepressed individuals. In addition, the added value of actigraphy-based measures was examined.

Methods

We used data from 2 samples to develop and validate prediction models. The development data set included 14 days of ESM and continuous actigraphy of currently depressed (n=43) and nondepressed individuals (n=82). The validation data set included 30 days of ESM and continuous actigraphy of currently depressed (n=27) and nondepressed individuals (n=27). Backward stepwise logistic regression analysis was applied to build the prediction models. Performance of the models was assessed with goodness-of-fit indices, calibration curves, and discriminative ability (area under the receiver operating characteristic curve [AUC]).

Results

In the development data set, the discriminative ability was good for the actigraphy model (AUC=0.790) and excellent for both the ESM (AUC=0.991) and the combined-domains model (AUC=0.993). In the validation data set, the discriminative ability was reasonable for the actigraphy model (AUC=0.648) and excellent for both the ESM (AUC=0.891) and the combined-domains model (AUC=0.892).

Conclusions

ESM is a good diagnostic predictor and is easy to calculate, and it therefore holds promise for implementation in clinical practice. Actigraphy shows no added value to ESM as a diagnostic predictor but might still be useful when ESM use is restricted.

SUBMITTER: Minaeva O

PROVIDER: S-EPMC7894744 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Screening for Depression in Daily Life: Development and External Validation of a Prediction Model Based on Actigraphy and Experience Sampling Method.

Minaeva Olga O Riese Harriëtte H Lamers Femke F Antypa Niki N Wichers Marieke M Booij Sanne H SH

Journal of medical Internet research 20201201 12

<h4>Background</h4>In many countries, depressed individuals often first visit primary care settings for consultation, but a considerable number of clinically depressed patients remain unidentified. Introducing additional screening tools may facilitate the diagnostic process.<h4>Objective</h4>This study aimed to examine whether experience sampling method (ESM)-based measures of depressive affect and behaviors can discriminate depressed from nondepressed individuals. In addition, the added value o ...[more]

PMID: 33258783

Similar Datasets

Project description:ObjectivesTo validate the newly developed Daily Experience Sampling Questionnaire (DESQ) that measures affective subjective well-being (SWB). The DESQ is an end-of-day diary in which respondents retrospectively rate their SWB at six different, randomly determined moments; it is completed over 1 week. The DESQ shall provide an alternative or complementary approach to existing methods of near-time SWB measurement (experience sampling, Day Reconstruction Method). The primary research objective was to determine criterion validity of the DESQ.DesignProspective, non-interventional study.SettingParticipants were recruited in Hamburg, Germany, at a specialised outpatient clinic (patients) and via different channels (healthy participants).Participants101 adults with diagnosed and stable psoriasis (46 women, 55 men); 105 adults without psoriasis (49 women, 56 men).Primary and secondary outcome measuresParticipants completed the DESQ for 3 weeks. In weeks 2 and 3, they also performed experience sampling. Criterion validity was determined by weekwise intraclass correlations (ICC) between both methods. Sensitivity to change was determined by the correlation between changes in both methods from weeks 2 to 3. For convergent validity, related concepts such as life satisfaction were measured. Retest reliability was determined using DESQ values of weeks 2 and 3.ResultsCriterion validity was excellent (ICC: patients=0.86, 95% CI 0.81 to 0.91; healthy participants=0.86, 95% CI 0.79 to 0.91). Sensitivity to change was r=0.57 and r=0.56, respectively. Correlations with convergent criteria were mostly significant and higher in constructs more proximal to SWB. The ICC indicating retest reliability was 0.77 in patients (95% CI 0.68 to 0.84) and 0.81 in healthy participants (95% CI 0.73 to 0.86).ConclusionsThe DESQ is a valid, reliable and feasible instrument for SWB measurement in people with psoriasis and healthy people. Its approach of end-of-day evaluations of single moments may also lend itself to the measurement of other highly time-variant constructs such as pain, fatigue or depression.

Project description:We propose a cost-effective sampling design and estimating procedure for validating personal risk models using right-censored cohort data. Validation involves using each subject's covariates, as ascertained at cohort entry, in a risk model (specified independently of the data) to assign him/her a probability of an adverse outcome within a future time period. Subjects are then grouped according to the magnitudes of their assigned risks, and within each group, the mean assigned risk is compared with the probability of outcome occurrence as estimated using the follow-up data. Such validation presents two complications. First, in the presence of right-censoring, estimating the probability of developing the outcomes before death requires competing risk analysis. Second, for rare outcomes, validation using the full cohort requires assembling covariates and assigning risks to thousands of subjects. This can be costly when some covariates involve analyzing biological specimens. A two-stage sampling design addresses this problem by assembling covariates and assigning risks only to those subjects most informative for estimating key parameters. We use this design to estimate the outcome probabilities needed to evaluate model performance and we provide theoretical and bootstrap estimates of their variances. We also describe how to choose two-stage designs with minimal efficiency loss for a parameter of interest when the quantities determining optimality are unknown at the time of design. We illustrate these methods by using subjects in the California Teachers Study to validate ovarian cancer risk models. We find that a design with optimal efficiency for one performance parameter need not be so for others, and trade-offs will be required. A two-stage design that samples all outcome-positive subjects and more outcome-negative than censored subjects will perform well in most circumstances. The methods are implemented in Risk Model Assessment Program, an R program freely available at http://med.stanford.edu/epidemiology/two-stage.html.

Project description:Background and objectivesThe International IgA Nephropathy Network recently developed and externally validated two models to predict the risk of progression of IgA nephropathy: full models without and with race. This study sought to externally validate the International IgA Nephropathy Prediction Tool in a large, independent, and contemporary cohort in China.Design, setting, participants, & measurementsWe included 1373 patients with biopsy-confirmed primary IgA nephropathy from The First Affiliated Hospital of Zhengzhou University from January 2012 to May 2018 and calculated predicted risks for each patient. The outcomes of interest were a 50% decline in eGFR or kidney failure. We assessed the performance of both models using discrimination (concordance statistics and Kaplan-Meier curves between subgroups), calibration (calibration plots), reclassification (net reclassification improvement and integrated discrimination improvement), and clinical utility (decision curve analysis).ResultsThe median follow-up was 29 months (interquartile range, 21-43 months; range, 1-95 months), and 186 (14%) patients reached the kidney outcomes of interest. Both models showed excellent discrimination (concordance statistics >0.85 and well separated survival curves). Overall, the full model without race generally underestimated the risk of primary outcome, whereas the full model with race was well calibrated for predicting 5-year risk. Compared with the full model without race, the full model with race had significant improvement in reclassification, as assessed by the net reclassification improvement (0.49; 95% confidence interval, 0.41 to 0.59) and integrated discrimination improvement (0.06; 95% confidence interval, 0.04 to 0.08). Decision curve analysis showed that both full models had a higher net benefit than default strategies, and the model with race performed better.ConclusionsIn this study, both full models demonstrated remarkable discrimination, acceptable calibration, and satisfactory clinical utility. The relatively short follow-up time may have limited the validation of these models.

Project description:IntroductionExternal validation studies are essential to study the generalizability of prediction models. Recently a permutation test, focusing on discrimination as quantified by the c-statistic, was proposed to judge whether a prediction model is transportable to a new setting. We aimed to evaluate this test and compare it to previously proposed procedures to judge any changes in c-statistic from development to external validation setting.MethodsWe compared the use of the permutation test to the use of benchmark values of the c-statistic following from a previously proposed framework to judge transportability of a prediction model. In a simulation study we developed a prediction model with logistic regression on a development set and validated them in the validation set. We concentrated on two scenarios: 1) the case-mix was more heterogeneous and predictor effects were weaker in the validation set compared to the development set, and 2) the case-mix was less heterogeneous in the validation set and predictor effects were identical in the validation and development set. Furthermore we illustrated the methods in a case study using 15 datasets of patients suffering from traumatic brain injury.ResultsThe permutation test indicated that the validation and development set were homogenous in scenario 1 (in almost all simulated samples) and heterogeneous in scenario 2 (in 17%-39% of simulated samples). Previously proposed benchmark values of the c-statistic and the standard deviation of the linear predictors correctly pointed at the more heterogeneous case-mix in scenario 1 and the less heterogeneous case-mix in scenario 2.ConclusionThe recently proposed permutation test may provide misleading results when externally validating prediction models in the presence of case-mix differences between the development and validation population. To correctly interpret the c-statistic found at external validation it is crucial to disentangle case-mix differences from incorrect regression coefficients.

Dataset Information

Screening for Depression in Daily Life: Development and External Validation of a Prediction Model Based on Actigraphy and Experience Sampling Method.

Background

Objective

Methods

Results

Conclusions

Publications

Screening for Depression in Daily Life: Development and External Validation of a Prediction Model Based on Actigraphy and Experience Sampling Method.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets