Dataset Information

Nonparametric variable importance assessment using machine learning techniques.

ABSTRACT: In a regression setting, it is often of interest to quantify the importance of various features in predicting the response. Commonly, the variable importance measure used is determined by the regression technique employed. For this reason, practitioners often only resort to one of a few regression techniques for which a variable importance measure is naturally defined. Unfortunately, these regression techniques are often suboptimal for predicting the response. Additionally, because the variable importance measures native to different regression techniques generally have a different interpretation, comparisons across techniques can be difficult. In this work, we study a variable importance measure that can be used with any regression technique, and whose interpretation is agnostic to the technique used. This measure is a property of the true data-generating mechanism. Specifically, we discuss a generalization of the analysis of variance variable importance measure and discuss how it facilitates the use of machine learning techniques to flexibly estimate the variable importance of a single feature or group of features. The importance of each feature or group of features in the data can then be described individually, using this measure. We describe how to construct an efficient estimator of this measure as well as a valid confidence interval. Through simulations, we show that our proposal has good practical operating characteristics, and we illustrate its use with data from a study of risk factors for cardiovascular disease in South Africa.

SUBMITTER: Williamson BD

PROVIDER: S-EPMC7946807 | biostudies-literature | 2021 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Nonparametric variable importance assessment using machine learning techniques.

Williamson Brian D BD Gilbert Peter B PB Carone Marco M Simon Noah N

Biometrics 20201208 1

In a regression setting, it is often of interest to quantify the importance of various features in predicting the response. Commonly, the variable importance measure used is determined by the regression technique employed. For this reason, practitioners often only resort to one of a few regression techniques for which a variable importance measure is naturally defined. Unfortunately, these regression techniques are often suboptimal for predicting the response. Additionally, because the variable ...[more]

PMID: 33043428

Similar Datasets

Project description:ImportanceCompetence in cataract surgery is a public health necessity, and videos of cataract surgery are routinely available to educators and trainees but currently are of limited use in training. Machine learning and deep learning techniques can yield tools that efficiently segment videos of cataract surgery into constituent phases for subsequent automated skill assessment and feedback.ObjectiveTo evaluate machine learning and deep learning algorithms for automated phase classification of manually presegmented phases in videos of cataract surgery.Design, setting, and participantsThis was a cross-sectional study using a data set of videos from a convenience sample of 100 cataract procedures performed by faculty and trainee surgeons in an ophthalmology residency program from July 2011 to December 2017. Demographic characteristics for surgeons and patients were not captured. Ten standard labels in the procedure and 14 instruments used during surgery were manually annotated, which served as the ground truth.ExposuresFive algorithms with different input data: (1) a support vector machine input with cross-sectional instrument label data; (2) a recurrent neural network (RNN) input with a time series of instrument labels; (3) a convolutional neural network (CNN) input with cross-sectional image data; (4) a CNN-RNN input with a time series of images; and (5) a CNN-RNN input with time series of images and instrument labels. Each algorithm was evaluated with 5-fold cross-validation.Main outcomes and measuresAccuracy, area under the receiver operating characteristic curve, sensitivity, specificity, and precision.ResultsUnweighted accuracy for the 5 algorithms ranged between 0.915 and 0.959. Area under the receiver operating characteristic curve for the 5 algorithms ranged between 0.712 and 0.773, with small differences among them. The area under the receiver operating characteristic curve for the image-only CNN-RNN (0.752) was significantly greater than that of the CNN with cross-sectional image data (0.712) (difference, -0.040; 95% CI, -0.049 to -0.033) and the CNN-RNN with images and instrument labels (0.737) (difference, 0.016; 95% CI, 0.014 to 0.018). While specificity was uniformly high for all phases with all 5 algorithms (range, 0.877 to 0.999), sensitivity ranged between 0.005 (95% CI, 0.000 to 0.015) for the support vector machine for wound closure (corneal hydration) and 0.974 (95% CI, 0.957 to 0.991) for the RNN for main incision. Precision ranged between 0.283 and 0.963.Conclusions and relevanceTime series modeling of instrument labels and video images using deep learning techniques may yield potentially useful tools for the automated detection of phases in cataract surgery procedures.

Project description:Importance For the assessment of optimum treatment timing in dentofacial orthopedics, understanding the growth process is of paramount importance. The evaluation of skeletal maturity based on study of the morphology of the cervical vertebrae has been devised to minimize radiation exposure of a patient due to hand wrist radiography. Cervical vertebral maturation assessment (CVMA) predictions have been examined in the state-of-the-art machine learning techniques in the recent past which require more attention and validation by clinicians and practitioners. Objectives This paper aimed to answer the question “How are machine learning techniques being employed in studies concerning cervical vertebral maturation assessment using lateral cephalograms?” Method A systematic search through the available literature was performed for this work based upon the Population, Intervention, Comparison and Outcome (PICO) framework. Data sources, study selection, data extraction and synthesis The searches were performed in Ovid Medline, Embase, PubMed and Cochrane Central Register of Controlled Trials (CENTRAL) and Cochrane Database of Systematic Reviews (CDSR). A search of the grey literature was also performed in Google Scholar and OpenGrey. We also did a hand-searching in the Angle Orthodontist, Journal of Orthodontics and Craniofacial Research, Progress in Orthodontics, and the American Journal of Orthodontics and Dentofacial Orthopedics. References from the included articles were also searched. Main outcome(s) and measure(s), results A total of 25 papers which were assessed for full text, and 13 papers were included for the systematic review. The machine learning methods used were scrutinized according to their performance and comparison to human observers/experts. The accuracy of the models ranged between 60 and 90% or above, and satisfactory agreement and correlation with the human observers. Conclusions and relevance Machine learning models can be used for detection and classification of the cervical vertebrae maturation. In this systematic review (SR), the studies were summarized in terms of ML techniques applied, sample data, age range of sample and conventional method for CVMA, which showed that further studies with a uniform distribution of samples equally in stages of maturation and according to the gender is required for better training of the models in order to generalize the outputs for prolific use to target population.

Dataset Information

Nonparametric variable importance assessment using machine learning techniques.

Publications

Nonparametric variable importance assessment using machine learning techniques.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets