Dataset Information

A Systematic Methodology to Evaluate Prediction Models for Driving Style Classification.

ABSTRACT: Identifying driving styles using classification models with in-vehicle data can provide automated feedback to drivers on their driving behavior, particularly if they are driving safely. Although several classification models have been developed for this purpose, there is no consensus on which classifier performs better at identifying driving styles. Therefore, more research is needed to evaluate classification models by comparing performance metrics. In this paper, a data-driven machine-learning methodology for classifying driving styles is introduced. This methodology is grounded in well-established machine-learning (ML) methods and literature related to driving-styles research. The methodology is illustrated through a study involving data collected from 50 drivers from two different cities in a naturalistic setting. Five features were extracted from the raw data. Fifteen experts were involved in the data labeling to derive the ground truth of the dataset. The dataset fed five different models (Support Vector Machines (SVM), Artificial Neural Networks (ANN), fuzzy logic, k-Nearest Neighbor (kNN), and Random Forests (RF)). These models were evaluated in terms of a set of performance metrics and statistical tests. The experimental results from performance metrics showed that SVM outperformed the other four models, achieving an average accuracy of 0.96, F1-Score of 0.9595, Area Under the Curve (AUC) of 0.9730, and Kappa of 0.9375. In addition, Wilcoxon tests indicated that ANN predicts differently to the other four models. These promising results demonstrate that the proposed methodology may support researchers in making informed decisions about which ML model performs better for driving-styles classification.

SUBMITTER: Silva I

PROVIDER: S-EPMC7146739 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Systematic Methodology to Evaluate Prediction Models for Driving Style Classification.

Silva Iván I Eugenio Naranjo José J

Sensors (Basel, Switzerland) 20200318 6

Identifying driving styles using classification models with in-vehicle data can provide automated feedback to drivers on their driving behavior, particularly if they are driving safely. Although several classification models have been developed for this purpose, there is no consensus on which classifier performs better at identifying driving styles. Therefore, more research is needed to evaluate classification models by comparing performance metrics. In this paper, a data-driven machine-learning ...[more]

PMID: 32197384

Similar Datasets

Project description:Background: The number of older adults in the United States will double by 2056. Additionally, the number of licensed drivers will increase along with extended driving-life expectancy. Motor vehicle crashes are a leading cause of injury and death in older adults. Alzheimer's disease (AD) also negatively impacts driving ability and increases crash risk. Conventional methods to evaluate driving ability are limited in predicting decline among older adults. Innovations in GPS hardware and software can monitor driving behavior in the actual environments people drive in. Commercial off-the-shelf (COTS) devices are affordable, easy to install and capture large volumes of data in real-time. However, adapting these methodologies for research can be challenging. This study sought to adapt a COTS device and determine an interval that produced accurate data on the actual route driven for use in future studies involving older adults with and without AD. Methods: Three subjects drove a single course in different vehicles at different intervals (30, 60 and 120 seconds), at different times of day, morning (9:00-11:59AM), afternoon (2:00-5:00PM) and night (7:00-10pm). The nine datasets were examined to determine the optimal collection interval. Results: Compared to the 120-second and 60-second intervals, the 30-second interval was optimal in capturing the actual route driven along with the lowest number of incorrect paths and affordability weighing considerations for data storage and curation. Discussion: Use of COTS devices offers minimal installation efforts, unobtrusive monitoring and discreet data extraction. However, these devices require strict protocols and controlled testing for adoption into research paradigms. After reliability and validity testing, these devices may provide valuable insight into daily driving behaviors and intraindividual change over time for populations of older adults with and without AD. Data can be aggregated over time to look at changes or adverse events and ascertain if decline in performance is occurring.

Project description:BACKGROUND:Clinical prediction models are widely used to guide medical advice and therapeutic interventions. Asthma is one of the most common chronic diseases globally and is characterised by acute deteriorations. These exacerbations are largely preventable, so there is interest in using clinical prediction models in this area. The objective of this review was to identify studies which have developed such models, determine whether consistent and appropriate methodology was used and whether statistically reliable prognostic models exist. METHODS:We searched online databases MEDLINE (1948 onwards), CINAHL Plus (1937 onwards), The Cochrane Library, Web of Science (1898 onwards) and ClinicalTrials.gov, using index terms relating to asthma and prognosis. Data was extracted and assessment of quality was based on GRADE and an early version of PROBAST (Prediction study Risk of Bias Assessment Tool). A meta-analysis of the discrimination and calibration measures was carried out to determine overall performance across models. RESULTS:Ten unique prognostic models were identified. GRADE identified moderate risk of bias in two of the studies, but more detailed quality assessment via PROBAST highlighted that most models were developed using highly selected and small datasets, incompletely recorded predictors and outcomes, and incomplete methodology. None of the identified models modelled recurrent exacerbations, instead favouring either presence/absence of an event, or time to first or specified event. Preferred methodologies were logistic regression and Cox proportional hazards regression. The overall pooled c-statistic was 0.77 (95% confidence interval 0.73 to 0.80), though individually some models performed no better than chance. The meta-analysis had an I2 value of 99.75% indicating a high amount of heterogeneity between studies. The majority of studies were small and did not include internal or external validation, therefore the individual performance measures are likely to be optimistic. CONCLUSIONS:Current prognostic models for asthma exacerbations are heterogeneous in methodology, but reported c-statistics suggest a clinically useful model could be created. Studies were consistent in lacking robust validation and in not modelling serial events. Further research is required with respect to incorporating recurrent events, and to externally validate tools in large representative populations to demonstrate the generalizability of published results.

Project description:Driving style is an important factor in driving psychology, which is useful for effectively evaluating driving behaviors. Exploring driving style in a Chinese sample could help understand the overall situation of Chinese drivers. This study aims to develop a measurement of driving style fit for the Chinese situation and to validate the validity and reliability of this scale. In this study, 296 drivers from China completed the Chinese version of the multidimensional driving style inventory (MDSI), the Big Five Inventory (BFI), and the Driver Behavior Questionnaire (DBQ) as well as several questions about socio-demographic information. After testing the assumed structure by a confirmatory factor analysis, and adjusting the structure, a brief version of Chinese version of MDSI with twelve items categorized into four driving styles, namely, risky style, angry- high-velocity style, careful style and anxious style was revised. The validity and reliability of the scales were acceptable. The results showed that driving styles were closely correlated with self-reported driving behaviors. Specifically, risky style, angry- high-velocity style, and anxious style are all positively associated with dangerous driving behaviors. Meanwhile, careful style was positively associated with positive driving behaviors and negatively correlated with dangerous driving behaviors. Anxious was also found to be negatively associated with fines. For personality, we found a positive relationship between risky, angry- high-velocity, careful and anxious styles and the personality traits that often have negative effects on driving, such as extraversion and neuroticism. Meanwhile, these three styles were negatively correlated with conscientiousness and agreeableness in general. In addition, careful style was positively correlated with conscientiousness and agreeableness. The current study shows convincing evidence for the validity and reliability of the brief MDSI-C and develops a useful tool for identify the driving style of Chinese drivers for future research and relevant departments of road safety.

Project description:BackgroundThere has been a rapid growth in the publication of new prediction models relevant to child and adolescent mental health. However, before their implementation into clinical services, it is necessary to appraise the quality of their methods and reporting. We conducted a systematic review of new prediction models in child and adolescent mental health, and examined their development and validation.MethodWe searched five databases for studies developing or validating multivariable prediction models for individuals aged 18 years old or younger from 1 January 2018 to 18 February 2021. Quality of reporting was assessed using the Transparent Reporting of a multivariable prediction models for Individual Prognosis Or Diagnosis checklist, and quality of methodology using items based on expert guidance and the PROBAST tool.ResultsWe identified 100 eligible studies: 41 developing a new prediction model, 48 validating an existing model and 11 that included both development and validation. Most publications (k = 75) reported a model discrimination measure, while 26 investigations reported calibration. Of 52 new prediction models, six (12%) were for suicidal outcomes, 18 (35%) for future diagnosis, five (10%) for child maltreatment. Other outcomes included violence, crime, and functional outcomes. Eleven new models (21%) were developed for use in high-risk populations. Of development studies, around a third were sufficiently statistically powered (k = 16%, 31%), while this was lower for validation investigations (k = 12, 25%). In terms of performance, the discrimination (as measured by the C-statistic) for new models ranged from 0.57 for a tool predicting ADHD diagnosis in an external validation sample to 0.99 for a machine learning model predicting foster care permanency.ConclusionsAlthough some tools have recently been developed for child and adolescent mental health for prognosis and child maltreatment, none can be currently recommended for clinical practice due to a combination of methodological limitations and poor model performance. New work needs to use ensure sufficient sample sizes, representative samples, and testing of model calibration.

Project description:Background/Objectives: Road tests and driving simulators are most commonly used in research studies and clinical evaluations of older drivers. Our objective was to describe the process and associated challenges in adapting an existing, commercial, off-the-shelf (COTS), in-vehicle device for naturalistic, longitudinal research to better understand daily driving behavior in older drivers. Design: The Azuga G2 Tracking Device TM was installed in each participant's vehicle, and we collected data over 5 months (speed, latitude/longitude) every 30-seconds when the vehicle was driven. Setting: The Knight Alzheimer's Disease Research Center at Washington University School of Medicine. Participants: Five individuals enrolled in a larger, longitudinal study assessing preclinical Alzheimer disease and driving performance. Participants were aged 65+ years and had normal cognition. Measurements: Spatial components included Primary Location(s), Driving Areas, Mean Centers and Unique Destinations. Temporal components included number of trips taken during different times of the day. Behavioral components included number of hard braking, speeding and sudden acceleration events. Methods: Individual 30-second observations, each comprising one breadcrumb, and trip-level data were collected and analyzed in R and ArcGIS. Results: Primary locations were confirmed to be 100% accurate when compared to known addresses. Based on the locations of the breadcrumbs, we were able to successfully identify frequently visited locations and general travel patterns. Based on the reported time from the breadcrumbs, we could assess number of trips driven in daylight vs. night. Data on additional events while driving allowed us to compute the number of adverse driving alerts over the course of the 5-month period. Conclusions: Compared to cameras and highly instrumented vehicle in other naturalistic studies, the compact COTS device was quickly installed and transmitted high volumes of data. Driving Profiles for older adults can be created and compared month-to-month or year-to-year, allowing researchers to identify changes in driving patterns that are unavailable in controlled conditions.

Dataset Information

A Systematic Methodology to Evaluate Prediction Models for Driving Style Classification.

Publications

A Systematic Methodology to Evaluate Prediction Models for Driving Style Classification.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets