Dataset Information

A Tool for Automatic Scoring of Spelling Performance.

ABSTRACT: Purpose The evaluation of spelling performance in aphasia reveals deficits in written language and can facilitate the design of targeted writing treatments. Nevertheless, manual scoring of spelling performance is time-consuming, laborious, and error prone. We propose a novel method based on the use of distance metrics to automatically score spelling. This study compares six automatic distance metrics to identify the metric that best corresponds to the gold standard-manual scoring-using data from manually obtained spelling scores from individuals with primary progressive aphasia. Method Three thousand five hundred forty word and nonword spelling productions from 42 individuals with primary progressive aphasia were scored manually. The gold standard-the manual scores-were compared to scores from six automated distance metrics: sequence matcher ratio, Damerau-Levenshtein distance, normalized Damerau-Levenshtein distance, Jaccard distance, Masi distance, and Jaro-Winkler similarity distance. We evaluated each distance metric based on its correlation with the manual spelling score. Results All automatic distance scores had high correlation with the manual method for both words and nonwords. The normalized Damerau-Levenshtein distance provided the highest correlation with the manual scoring for both words (r_s = .99) and nonwords (r_s = .95). Conclusions The high correlation between the automated and manual methods suggests that automatic spelling scoring constitutes a quick and objective approach that can reliably substitute the existing manual and time-consuming spelling scoring process, an important asset for both researchers and clinicians.

SUBMITTER: Themistocleous C

PROVIDER: S-EPMC8608207 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:PurposePoint-of-care ultrasound (POCUS) is a sensitive, safe, and efficient tool used in many clinical settings and is an essential part of medical education in the United States. Numerous studies present improved diagnostic performances and positive clinical outcomes among POCUS users. However, others stress the degree to which the modality is user-dependent, rendering high-quality POCUS training necessary in medical education. In this study, the authors aimed to investigate the potential of an artificial intelligence (AI) based quality indicator tool as a teaching device for cardiac POCUS performance.MethodsThe authors integrated the quality indicator tool into the pre-clinical cardiac ultrasound course for 4th-year medical students and analyzed their performances. The analysis included 60 students who were assigned to one of two groups as follows: the intervention group using the AI-based quality indicator tool and the control group. Quality indicator users utilized the tool during both the course and the final test. At the end of the course, the authors tested the standard echocardiographic views, and an experienced clinician blindly graded the recorded clips. Results were analyzed and compared between the groups.ResultsThe results showed an advantage in quality indictor users' median overall scores (P = 0.002) with a relative risk of 2.3 (95% CI: 1.10, 4.93, P = 0.03) for obtaining correct cardiac views. In addition, quality indicator users also had a statistically significant advantage in the overall image quality in various cardiac views.ConclusionsThe AI-based quality indicator improved cardiac ultrasound performances among medical students who were trained with it compared to the control group, even in cardiac views in which the indicator was inactive. Performance scores, as well as image quality, were better in the AI-based group. Such tools can potentially enhance ultrasound training, warranting the expansion of the application to more views and prompting further studies on long-term learning effects.

Dataset Information

A Tool for Automatic Scoring of Spelling Performance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets