Dataset Information

Using Machine Learning and Structural Neuroimaging to Detect First Episode Psychosis: Reconsidering the Evidence.

ABSTRACT: Despite the high level of interest in the use of machine learning (ML) and neuroimaging to detect psychosis at the individual level, the reliability of the findings is unclear due to potential methodological issues that may have inflated the existing literature. This study aimed to elucidate the extent to which the application of ML to neuroanatomical data allows detection of first episode psychosis (FEP), while putting in place methodological precautions to avoid overoptimistic results. We tested both traditional ML and an emerging approach known as deep learning (DL) using 3 feature sets of interest: (1) surface-based regional volumes and cortical thickness, (2) voxel-based gray matter volume (GMV) and (3) voxel-based cortical thickness (VBCT). To assess the reliability of the findings, we repeated all analyses in 5 independent datasets, totaling 956 participants (514 FEP and 444 within-site matched controls). The performance was assessed via nested cross-validation (CV) and cross-site CV. Accuracies ranged from 50% to 70% for surfaced-based features; from 50% to 63% for GMV; and from 51% to 68% for VBCT. The best accuracies (70%) were achieved when DL was applied to surface-based features; however, these models generalized poorly to other sites. Findings from this study suggest that, when methodological precautions are adopted to avoid overoptimistic results, detection of individuals in the early stages of psychosis is more challenging than originally thought. In light of this, we argue that the current evidence for the diagnostic value of ML and structural neuroimaging should be reconsidered toward a more cautious interpretation.

SUBMITTER: Vieira S

PROVIDER: S-EPMC6942152 | biostudies-literature | 2020 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using Machine Learning and Structural Neuroimaging to Detect First Episode Psychosis: Reconsidering the Evidence.

Vieira Sandra S Gong Qi-Yong QY Pinaya Walter H L WHL Scarpazza Cristina C Tognin Stefania S Crespo-Facorro Benedicto B Tordesillas-Gutierrez Diana D Ortiz-García Victor V Setien-Suero Esther E Scheepers Floortje E FE Van Haren Neeltje E M NEM Marques Tiago R TR Murray Robin M RM David Anthony A Dazzan Paola P McGuire Philip P Mechelli Andrea A

Schizophrenia bulletin 20200101 1

Despite the high level of interest in the use of machine learning (ML) and neuroimaging to detect psychosis at the individual level, the reliability of the findings is unclear due to potential methodological issues that may have inflated the existing literature. This study aimed to elucidate the extent to which the application of ML to neuroanatomical data allows detection of first episode psychosis (FEP), while putting in place methodological precautions to avoid overoptimistic results. We test ...[more]

PMID: 30809667

Similar Datasets

Project description:BACKGROUND:Early illness course correlates with long-term outcome in psychosis. Accurate prediction could allow more focused intervention. Earlier intervention corresponds to significantly better symptomatic and functional outcomes. Our study objective is to use routinely collected baseline demographic and clinical characteristics to predict employment, education or training (EET) status, and symptom remission in patients with first episode psychosis (FEP) at one-year. METHODS AND FINDINGS:83 FEP patients were recruited from National Health Service (NHS) Glasgow between 2011 and 2014 to a 24-month prospective cohort study with regular assessment of demographic and psychometric measures. An external independent cohort of 79 FEP patients were recruited from NHS Glasgow and Edinburgh during a 12-month study between 2006 and 2009. Elastic net regularised logistic regression models were built to predict binary EET status, period and point remission outcomes at one-year on 83 Glasgow patients (training dataset). Models were externally validated on an independent dataset of 79 patients from Glasgow and Edinburgh (validation dataset). Only baseline predictors shared across both cohorts were made available for model training and validation. After excluding participants with missing outcomes, models were built on the training dataset for EET status, period and point remission outcomes and externally validated on the validation dataset. Models predicted EET status, period and point remission with receiver operating curve (ROC) area under the curve (AUC) performances of 0.876 (95%CI: 0.864, 0.887), 0.630 (95%CI: 0.612, 0.647) and 0.652 (95%CI: 0.635, 0.670) respectively. Positive predictors of EET included baseline EET and living with spouse/children. Negative predictors included higher PANSS suspiciousness, hostility and delusions scores. Positive predictors for symptom remission included living with spouse/children, and affective symptoms on the Positive and Negative Syndrome Scale (PANSS). Negative predictors of remission included passive social withdrawal symptoms on PANSS. A key limitation of this study is the small sample size (n) relative to the number of predictors (p), whereby p approaches n. The use of elastic net regularised regression rather than ordinary least squares regression helped circumvent this difficulty. Further, we did not have information for biological and additional social variables, such as nicotine dependence, which observational studies have linked to outcomes in psychosis. CONCLUSIONS AND RELEVANCE:Using advanced statistical machine learning techniques, we provide the first externally validated evidence, in a temporally and geographically independent cohort, for the ability to predict one-year EET status and symptom remission in individual FEP patients.

Project description:ImportanceThere is an absence of mortality risk assessment tools in first-episode psychosis (FEP) that could enable personalized interventions.ObjectiveTo examine the feasibility of machine learning (ML) in discerning mortality risk in FEP and to assess whether such risk predictions can inform pharmacotherapy choices.Design, setting, and participantsIn this prognostic study, Swedish nationwide cohort data (from July 1, 2006, to December 31, 2021) were harnessed for model development and validation. Finnish cohort data (from January 1, 1998, to December 31, 2017) were used for external validation. Data analyses were completed between December 2022 and December 2023.Main outcomes and measuresFifty-one nationwide register variables, encompassing demographics and clinical and work-related histories, were subjected to ML to predict future mortality risk. The ML model's performance was evaluated by calculating the area under the receiver operating characteristic curve (AUROC). The comparative effectiveness of pharmacotherapies in patients was assessed and was stratified by the ML model to those with predicted high mortality risk (vs low risk), using the between-individual hazard ratio (HR). The 5 most important variables were then identified and a model was retrained using these variables in the discovery sample.ResultsThis study included 24 052 Swedish participants (20 000 in the discovery sample and 4052 in the validation sample) and 1490 Finnish participants (in the validation sample). Swedish participants had a mean (SD) age of 29.1 (8.1) years, 62.1% were men, and 418 died with 2 years. Finnish participants had a mean (SD) age of 29.7 (8.0) years, 61.7% were men, and 31 died within 2 years. The discovery sample achieved an AUROC of 0.71 (95% CI, 0.68-0.74) for 2-year mortality prediction. Using the 5 most important variables (ie, the top 10% [substance use comorbidities, first hospitalization duration due to FEP, male sex, prior somatic hospitalizations, and age]), the final model resulted in an AUROC of 0.70 (95% CI, 0.63-0.76) in the Swedish sample and 0.67 (95% CI, 0.56-0.78) in the Finnish sample. Individuals with predicted high mortality risk had an elevated 15-year risk in the Swedish sample (HR, 3.77 [95% CI, 2.92-4.88]) and an elevated 20-year risk in the Finnish sample (HR, 3.72 [95% CI, 2.67-5.18]). For those with predicted high mortality risk, long-acting injectable antipsychotics (HR, 0.45 [95% CI, 0.23-0.88]) and mood stabilizers (HR, 0.64 [95% CI, 0.46-0.90]) were associated with decreased mortality risk. Conversely, for those predicted to survive, only oral aripiprazole (HR, 0.38 [95% CI, 0.20-0.69]) and risperidone (HR, 0.38 [95% CI, 0.18-0.82]) were associated with decreased mortality risk.Conclusions and relevanceIn this prognostic study, an ML-based model was developed and validated to predict mortality risk in FEP. These findings may help to develop personalized interventions to mitigate mortality risk in FEP.

Dataset Information

Using Machine Learning and Structural Neuroimaging to Detect First Episode Psychosis: Reconsidering the Evidence.

Publications

Using Machine Learning and Structural Neuroimaging to Detect First Episode Psychosis: Reconsidering the Evidence.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets