A systematic review of major evaluation metrics for simulator-based automatic assessment of driving after stroke
Ontology highlight
ABSTRACT:
SUBMITTER: Taveekitworachai P
PROVIDER: S-EPMC11252877 | biostudies-literature | 2024 Jun
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:There is a myriad of methodologies to assess driving performance after a stroke. These include psychometric tests, driving simulation, questionnaires, and/or road tests. Research-based driving simulators have emerged as a safe, convenient way to assess driving performance after a stroke. Such traditional research simulators are useful in recreating street traffic scenarios, but are often expensive, with limited physics models and graphics rendering. In contrast, racing simulators developed for motorsport professionals and enthusiasts offer high levels of realism, run on consumer-grade hardware, and can provide rich telemetric data. However, most offer limited simulation of traffic scenarios. This pilot study compares the feasibility of research simulation and racing simulation in a sample with minor stroke. We determine that the racing simulator is tolerated well in subjects with a minor stroke. There were correlations between research and racing simulator outcomes with psychometric tests associated with driving performance, such as the Trails Making Test Part A, Snellgrove Maze Task, and the Motricity Index. We found correlations between measures of driving speed on a complex research simulator scenario and racing simulator lap time and maximum tires off track. Finally, we present two models, using outcomes from either the research or racing simulator, predicting road test failure as linked to a previously published fitness-to-drive calculator that uses psychometric screening.
| S-EPMC7345038 | biostudies-literature
Project description:This paper presents a novel approach for a low-cost simulator-based driving assessment system incorporating a speech-based assistant, using pre-generated messages from Generative AI to achieve real-time interaction during the assessment. Simulator-based assessment is a crucial apparatus in the research toolkit for various fields. Traditional assessment approaches, like on-road evaluation, though reliable, can be risky, costly, and inaccessible. Simulator-based assessment using stationary driving simulators offers a safer evaluation and can be tailored to specific needs. However, these simulators are often only available to research-focused institutions due to their cost. To address this issue, our study proposes a system with the aforementioned properties aiming to enhance drivers' situational awareness, and foster positive emotional states, i.e., high valence and medium arousal, while assessing participants to prevent subpar performers from proceeding to the next stages of assessment and/or rehabilitation. In addition, this study introduces the speech-based assistant which provides timely guidance adaptable to the ever-changing context of the driving environment and vehicle state. The study's preliminary outcomes reveal encouraging progress, highlighting improved driving performance and positive emotional states when participants are engaged with the assistant during the assessment.
| S-EPMC11381592 | biostudies-literature
Project description:Cardiovascular diseases are the leading cause of death, globally. Stroke and heart attacks account for more than 80% of cardiovascular disease-related deaths. To prevent patient mismanagement and potentially save lives, effective screening at an early stage is needed. Diagnosis is typically made using an electrocardiogram (ECG) analysis. However, ECG recordings are often corrupted by different types of noise, degrading the quality of the recording and making diagnosis more difficult. This paper reviews research on automatic ECG quality assessment techniques used in studies published from 2012-2022. The CinC11 Dataset is most often used for training and testing algorithms. Only one study tested its algorithm on people in real-time, but it did not specify the demographic data of the subjects. Most of the reviewed papers evaluated the quality of the ECG recordings per single lead. The accuracy of the algorithms reviewed in this paper range from 85.75% to 97.15%. More clarity on the research methods used is needed to improve the quality of automatic ECG quality assessment techniques and implement them in a clinical setting. This paper discusses the possible shortcomings in current research and provides recommendations on how to advance the field of automatic ECG quality assessment.
| S-EPMC9689601 | biostudies-literature
Project description: Not available
| S-EPMC11674257 | biostudies-literature
Project description:BackgroundThe available literature on mobile stroke units (MSU) has focused on clinical outcomes, rather than operational performance. Our objective was to establish normalized metrics and to conduct a meta-analysis of the current literature on MSU performance.MethodsOur MSU in upstate New York serves 741,000 people. We present prospectively collected, retrospectively analyzed data from the inception of our MSU in October of 2018, through March of 2021. Rates of transportation/dispatch and MSU utilization were reported. We also performed a meta-analysis using MEDLINE, SCOPUS, and Cochrane Library databases, calculating rates of tPA/dispatch, tPA-per-24-operational-hours ("per day"), mechanical thrombectomy (MT)/dispatch and MT/day.ResultsOur MSU was dispatched 1,719 times in 606 days (8.5 dispatches/24-operational-hours) and transported 324 patients (18.8%) to the hospital. Intravenous tPA was administered in 64 patients (3.7% of dispatches) and the rate of tPA/day was 0.317 (95% CI 0.150-0.567). MT was performed in 24 patients (1.4% of dispatches) for a MT/day rate of 0.119 (95% CI 0.074-0.163). The MSU was in use for 38,742 minutes out of 290,760 total available minutes (13.3% utilization rate). Our meta-analysis included 14 articles. Eight studies were included in the analysis of tPA/dispatch (342/5,862) for a rate of 7.2% (95% CI 4.8-9.5%, I2 = 92%) and 11 were included in the analysis of tPA/day (1,858/4,961) for a rate of 0.358 (95% CI 0.215-0.502, I2 = 99%). Seven studies were included for MT/dispatch (102/5,335) for a rate of 2.0% (95% CI 1.2-2.8%, I2 = 67%) and MT/day (103/1,249) for a rate of 0.092 (95% CI 0.046-0.138, I2 = 91%).ConclusionsIn this single institution retrospective study and meta-analysis, we outline the following operational metrics: tPA/dispatch, tPA/day, MT/dispatch, MT/day, and utilization rate. These metrics are useful for internal and external comparison for institutions with or considering developing mobile stroke programs.
| S-EPMC9124821 | biostudies-literature
Project description:The purpose of this study was to characterize the motion features of surgical devices associated with laparoscopic surgical competency and build an automatic skill-credential system in porcine cadaver organ simulation training. Participants performed tissue dissection around the aorta, dividing vascular pedicles after applying Hem-o-lok (tissue dissection task) and parenchymal closure of the kidney (suturing task). Movements of surgical devices were tracked by a motion capture (Mocap) system, and Mocap-metrics were compared according to the level of surgical experience (experts: ≥50 laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test and principal component analysis (PCA). Three machine-learning algorithms: support vector machine (SVM), PCA-SVM, and gradient boosting decision tree (GBDT), were utilized for discrimination of the surgical experience level. The accuracy of each model was evaluated by nested and repeated k-fold cross-validation. A total of 32 experts, 18 intermediates, and 20 novices participated in the present study. PCA revealed that efficiency-related metrics (e.g., path length) significantly contributed to PC 1 in both tasks. Regarding PC 2, speed-related metrics (e.g., velocity, acceleration, jerk) of right-hand devices largely contributed to the tissue dissection task, while those of left-hand devices did in the suturing task. Regarding the three-group discrimination, in the tissue dissection task, the GBDT method was superior to the other methods (median accuracy: 68.6%). In the suturing task, SVM and PCA-SVM methods were superior to the GBDT method (57.4 and 58.4%, respectively). Regarding the two-group discrimination (experts vs. intermediates/novices), the GBDT method resulted in a median accuracy of 72.9% in the tissue dissection task, and, in the suturing task, the PCA-SVM method resulted in a median accuracy of 69.2%. Overall, the mocap-based credential system using machine-learning classifiers provides a correct judgment rate of around 70% (two-group discrimination). Together with motion analysis and wet-lab training, simulation training could be a practical method for objectively assessing the surgical competence of trainees.
| S-EPMC9629630 | biostudies-literature
Project description:Background and objectivesElectronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition.MethodsFollowing Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist.ResultsThe search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported. The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality."ConclusionThere is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics, and reporting. Standardized processes may improve evaluation and comparison electronic health record systems and improve their development and implementation.
| S-EPMC9438452 | biostudies-literature
Project description:BackgroundAfter total hip replacement surgery, patients are eager to resume the activities of daily life, particularly driving. Most surgeons recommend waiting 6 weeks after surgery to resume driving; however, there is no evidence to indicate that patients cannot resume driving earlier.Questions/purposesOur purpose was to evaluate when in the recovery period following THA that patients regain or improve upon their preoperative braking reaction time, allowing them to safely resume driving.MethodsWe measured and compared pre- and postoperative braking reaction times of 90 patients from 3 different surgeons using a Fully Interactive Driving Simulator (Simulator Systems International, Tulsa, OK). We defined a return to safe braking reaction time as a return to a time value that is either equal to or less than the preoperative braking reaction time.ResultsPatients tested at 2 and 3 weeks after surgery had slower braking reaction times than preoperative times by an average of 0.069 and 0.009 s, respectively. At 4 weeks after surgery, however, patients improved their reaction times by 0.035 s (p = 0.0398). In addition, at 2, 3, and 4 weeks postoperatively, the results also demonstrated that patient less than 70 years of age recovered faster.ConclusionsBased upon the results of this study, most patients should be allowed to return to driving 4 weeks following minimally invasive primary total hip arthroplasty.
| S-EPMC4481257 | biostudies-literature
Project description:Co-produced practices and publications in the healthcare sector are gaining momentum, since they can be a useful tool in addressing the sustainability and resilience challenges of health systems. However, the investigation of positive and, mainly, negative outcomes is still confused and fragmented, and above all, a comprehensive knowledge of the metrics used to assess these outcomes is lacking. To fill this gap, this study aims to systematically review the extant literature to map the methods, tools and metrics used to empirically evaluate co-production in health services. The search took place in six databases: Scopus, Web of Science, Psych INFO, PubMed, Cochrane and CINAHL. A total of 2311 articles were screened and 203 articles were included in the analysis, according to PRISMA guidelines. Findings show that outcomes are mainly investigated through qualitative methods and from the lay actor or provider perspective. Moreover, the detailed categorisation of the quantitative measures found offers a multidimensional performance measurement system and highlights the impact areas where research is needed to develop and test new measures. Findings should also promote improvements in empirical data collection on the multiple faceted co-produced activities and spur the consciousness of the adoption of sustainable co-productive initiatives.
| S-EPMC8037812 | biostudies-literature
Project description:A growing body of work has examined the act of evaluating the quality of a musical performance. This article considers the domain of training evaluative skills in musicians, presenting assessment as a form of performance to be taught and demonstrating a gap in opportunities for trainees to develop evaluative skills within the heightened environments of live assessment scenarios. To address these needs, the concepts of Immersive Virtual Environments (IVEs) and distributed simulation are described, highlighting their use in training and research in other performance domains. Taking this model as a starting point, we present the Evaluation Simulator as a new tool to study and train performance evaluation. Potential applications of this prototype technology in pedagogical and research settings are then discussed.
| S-EPMC6465612 | biostudies-literature