Dataset Information

Interpretability and Class Imbalance in Prediction Models for Pain Volatility in Manage My Pain App Users: Analysis Using Feature Selection and Majority Voting Methods.

ABSTRACT: BACKGROUND:Pain volatility is an important factor in chronic pain experience and adaptation. Previously, we employed machine-learning methods to define and predict pain volatility levels from users of the Manage My Pain app. Reducing the number of features is important to help increase interpretability of such prediction models. Prediction results also need to be consolidated from multiple random subsamples to address the class imbalance issue. OBJECTIVE:This study aimed to: (1) increase the interpretability of previously developed pain volatility models by identifying the most important features that distinguish high from low volatility users; and (2) consolidate prediction results from models derived from multiple random subsamples while addressing the class imbalance issue. METHODS:A total of 132 features were extracted from the first month of app use to develop machine learning-based models for predicting pain volatility at the sixth month of app use. Three feature selection methods were applied to identify features that were significantly better predictors than other members of the large features set used for developing the prediction models: (1) Gini impurity criterion; (2) information gain criterion; and (3) Boruta. We then combined the three groups of important features determined by these algorithms to produce the final list of important features. Three machine learning methods were then employed to conduct prediction experiments using the selected important features: (1) logistic regression with ridge estimators; (2) logistic regression with least absolute shrinkage and selection operator; and (3) random forests. Multiple random under-sampling of the majority class was conducted to address class imbalance in the dataset. Subsequently, a majority voting approach was employed to consolidate prediction results from these multiple subsamples. The total number of users included in this study was 879, with a total number of 391,255 pain records. RESULTS:A threshold of 1.6 was established using clustering methods to differentiate between 2 classes: low volatility (n=694) and high volatility (n=185). The overall prediction accuracy is approximately 70% for both random forests and logistic regression models when using 132 features. Overall, 9 important features were identified using 3 feature selection methods. Of these 9 features, 2 are from the app use category and the other 7 are related to pain statistics. After consolidating models that were developed using random subsamples by majority voting, logistic regression models performed equally well using 132 or 9 features. Random forests performed better than logistic regression methods in predicting the high volatility class. The consolidated accuracy of random forests does not drop significantly (601/879; 68.4% vs 618/879; 70.3%) when only 9 important features are included in the prediction model. CONCLUSIONS:We employed feature selection methods to identify important features in predicting future pain volatility. To address class imbalance, we consolidated models that were developed using multiple random subsamples by majority voting. Reducing the number of features did not result in a significant decrease in the consolidated prediction accuracy.

SUBMITTER: Rahman QA

PROVIDER: S-EPMC6913759 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Interpretability and Class Imbalance in Prediction Models for Pain Volatility in Manage My Pain App Users: Analysis Using Feature Selection and Majority Voting Methods.

Rahman Quazi Abidur QA Janmohamed Tahir T Clarke Hance H Ritvo Paul P Heffernan Jane J Katz Joel J

JMIR medical informatics 20191120 4

<h4>Background</h4>Pain volatility is an important factor in chronic pain experience and adaptation. Previously, we employed machine-learning methods to define and predict pain volatility levels from users of the Manage My Pain app. Reducing the number of features is important to help increase interpretability of such prediction models. Prediction results also need to be consolidated from multiple random subsamples to address the class imbalance issue.<h4>Objective</h4>This study aimed to: (1) i ...[more]

PMID: 31746764

Similar Datasets

Project description:Pain is one of the most prevalent health-related concerns and is among the top 3 most common reasons for seeking medical help. Scientific publications of data collected from pain tracking and monitoring apps are important to help consumers and healthcare professionals select the right app for their use.The main objectives of this paper were to (1) discover user engagement patterns of the pain management app, Manage My Pain, using data mining methods; and (2) identify the association between several attributes characterizing individual users and their levels of engagement.User engagement was defined by 2 key features of the app: longevity (number of days between the first and last pain record) and number of records. Users were divided into 5 user engagement clusters employing the k-means clustering algorithm. Each cluster was characterized by 6 attributes: gender, age, number of pain conditions, number of medications, pain severity, and opioid use. Z tests and chi-square tests were used for analyzing categorical attributes. Effects of gender and cluster on numerical attributes were analyzed using 2-way analysis of variances (ANOVAs) followed up by pairwise comparisons using Tukey honest significant difference (HSD).The clustering process produced 5 clusters representing different levels of user engagement. The proportion of males and females was significantly different in 4 of the 5 clusters (all P ≤.03). The proportion of males was higher than females in users with relatively high longevity. Mean ages of users in 2 clusters with high longevity were higher than users from other 3 clusters (all P <.001). Overall, males were significantly older than females (P <.001). Across clusters, females reported more pain conditions than males (all P <.001). Users from highly engaged clusters reported taking more medication than less engaged users (all P <.001). Females reported taking a greater number of medications than males (P =.04). In 4 of 5 clusters, the percentage of males taking an opioid was significantly greater (all P ≤.05) than that of females. The proportion of males with mild pain was significantly higher than that of females in 3 clusters (all P ≤.008).Although most users of the app reported being female, male users were more likely to be highly engaged in the app. Users in the most engaged clusters self-reported a higher number of pain conditions, a higher number of current medications, and a higher incidence of opioid usage. The high engagement by males in these clusters does not appear to be driven by pain severity which may, in part, be the case for females. Use of a mobile pain app may be relatively more attractive to highly-engaged males than highly-engaged females, and to those with relatively more complex chronic pain problems.

Project description:IntroductionIn the field of personalized medicine, radiomics has shown its potential to support treatment decisions. However, the limited feature interpretability hampers its introduction into the clinics. Here, we propose a new methodology to create radiomics feature activation maps, which allows to identify the spatial-anatomical locations responsible for signature activation based on local radiomics. The feasibility of this technique will be studied for histological subtype differentiation (adenocarcinoma versus squamous cell carcinoma) in non-small cell lung cancer (NSCLC) using computed tomography (CT) radiomics.Materials and methodsPre-treatment CT scans were collected from a multi-centric Swiss trial (training, n=73, IIIA/N2 NSCLC, SAKK 16/00) and an independent cohort (validation, n=32, IIIA/N2/IIIB NSCLC). Based on the gross tumor volume (GTV), four peritumoral region of interests (ROI) were defined: lung_exterior (expansion into the lung), iso_exterior (expansion into lung and soft tissue), gradient (GTV border region), GTV+Rim (GTV and iso_exterior). For each ROI, 154 radiomic features were extracted using an in-house developed software implementation (Z-Rad, Python v2.7.14). Features robust against delineation variability served as an input for a multivariate logistic regression analysis. Model performance was quantified using the area under the receiver operating characteristic curve (AUC) and verified using five-fold cross validation and internal validation. Local radiomic features were extracted from the GTV+Rim ROI using non-overlapping 3x3x3 voxel patches previously marked as GTV or rim. A binary activation map was created for each patient using the median global feature value from the training. The ratios of activated/non-activated patches of GTV and rim regions were compared between histological subtypes (Wilcoxon test).ResultsIso_exterior, gradient, GTV+Rim showed good performances for histological subtype prediction (AUCtraining=0.68-0.72 and AUCvalidation=0.73-0.74) whereas GTV and lung_exterior models failed validation. GTV+Rim model feature activation maps showed that local texture feature distribution differed significantly between histological subtypes in the rim (p=0.0481) but not in the GTV (p=0.461).ConclusionIn this exploratory study, radiomics-based prediction of NSCLC histological subtypes was predominantly based on the peritumoral region indicating that radiomics activation maps can be useful for tracing back the spatial location of regions responsible for signature activation.

Project description:BACKGROUND: Identifying the regions associated with protein function is a singularly important task in the post-genomic era. Biological studies often identify functional enzyme residues by amino acid sequences, particularly when related structural information is unavailable. In some cases of protein superfamilies, functional residues are difficult to detect by current alignment tools or evolutionary strategies when phylogenetic relationships do not parallel their protein functions. The solution proposed in this study is Feature Amplified Voting Algorithm with Three-profile alignment (FAVAT). The core concept of FAVAT is to reveal the desired features of a target enzyme or protein by voting on three different property groups aligned by three-profile alignment method. Functional residues of a target protein can then be retrieved by FAVAT analysis. In this study, the amidohydrolase superfamily was an interesting case for verifying the proposed approach because it contains divergent enzymes and proteins. RESULTS: The FAVAT was used to identify critical residues of mammalian imidase, a member of the amidohydrolase superfamily. Members of this superfamily were first classified by their functional properties and sources of original organisms. After FAVAT analysis, candidate residues were identified and compared to a bacterial hydantoinase in which the crystal structure (1GKQ) has been fully elucidated. One modified lysine, three histidines and one aspartate were found to participate in the coordination of metal ions in the active site. The FAVAT analysis also redressed the misrecognition of metal coordinator Asp57 by the multiple sequence alignment (MSA) method. Several other amino acid residues known to be related to the function or structure of mammalian imidase were also identified. CONCLUSIONS: The FAVAT is shown to predict functionally important amino acids in amidohydrolase superfamily. This strategy effectively identifies functionally important residues by analyzing the discrepancy between the sequence and functional properties of related proteins in a superfamily, and it should be applicable to other protein families.

Dataset Information

Interpretability and Class Imbalance in Prediction Models for Pain Volatility in Manage My Pain App Users: Analysis Using Feature Selection and Majority Voting Methods.

Publications

Interpretability and Class Imbalance in Prediction Models for Pain Volatility in Manage My Pain App Users: Analysis Using Feature Selection and Majority Voting Methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets