Dataset Information

A Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography: System Development and Validation Study.

ABSTRACT:

Background

Hip fracture is the most common type of fracture in elderly individuals. Numerous deep learning (DL) algorithms for plain pelvic radiographs (PXRs) have been applied to improve the accuracy of hip fracture diagnosis. However, their efficacy is still undetermined.

Objective

The objective of this study is to develop and validate a human-algorithm integration (HAI) system to improve the accuracy of hip fracture diagnosis in a real clinical environment.

Methods

The HAI system with hip fracture detection ability was developed using a deep learning algorithm trained on trauma registry data and 3605 PXRs from August 2008 to December 2016. To compare their diagnostic performance before and after HAI system assistance using an independent testing dataset, 34 physicians were recruited. We analyzed the physicians' accuracy, sensitivity, specificity, and agreement with the algorithm; we also performed subgroup analyses according to physician specialty and experience. Furthermore, we applied the HAI system in the emergency departments of different hospitals to validate its value in the real world.

Results

With the support of the algorithm, which achieved 91% accuracy, the diagnostic performance of physicians was significantly improved in the independent testing dataset, as was revealed by the sensitivity (physician alone, median 95%; HAI, median 99%; P<.001), specificity (physician alone, median 90%; HAI, median 95%; P<.001), accuracy (physician alone, median 90%; HAI, median 96%; P<.001), and human-algorithm agreement [physician alone ?, median 0.69 (IQR 0.63-0.74); HAI ?, median 0.80 (IQR 0.76-0.82); P<.001. With the help of the HAI system, the primary physicians showed significant improvement in their diagnostic performance to levels comparable to those of consulting physicians, and both the experienced and less-experienced physicians benefited from the HAI system. After the HAI system had been applied in 3 departments for 5 months, 587 images were examined. The sensitivity, specificity, and accuracy of the HAI system for detecting hip fractures were 97%, 95.7%, and 96.08%, respectively.

Conclusions

HAI currently impacts health care, and integrating this technology into emergency departments is feasible. The developed HAI system can enhance physicians' hip fracture diagnostic performance.

SUBMITTER: Cheng CT

PROVIDER: S-EPMC7732715 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography: System Development and Validation Study.

Cheng Chi-Tung CT Chen Chih-Chi CC Cheng Fu-Jen FJ Chen Huan-Wu HW Su Yi-Siang YS Yeh Chun-Nan CN Chung I-Fang IF Liao Chien-Hung CH

JMIR medical informatics 20201127 11

<h4>Background</h4>Hip fracture is the most common type of fracture in elderly individuals. Numerous deep learning (DL) algorithms for plain pelvic radiographs (PXRs) have been applied to improve the accuracy of hip fracture diagnosis. However, their efficacy is still undetermined.<h4>Objective</h4>The objective of this study is to develop and validate a human-algorithm integration (HAI) system to improve the accuracy of hip fracture diagnosis in a real clinical environment.<h4>Methods</h4>The H ...[more]

PMID: 33245279

Similar Datasets

Project description:Background The ability to take care of oneself after hip fracture surgery is important for older adults. Various scales have been developed for evaluation of this ability, but a scale specifically focusing on hip fracture has not been developed. The aim of this study was to develop and validate a scale (Hip Fracture Surgery Self-Care Scale, HFS-SC) to evaluate self-care for older adults undergoing hip fracture surgery. Methods The scale was developed according to the guidelines by DeVellis. Initial items were derived from a literature review and individual interviews with 11 older adults who underwent hip fracture surgery. To confirm the suitability of the questions, a preliminary survey was conducted on 25 older adults. Psychometric testing was performed on 300 older adults 65 years old or over living at home after surgery for hip fracture. Psychometric properties of the scale were examined by content validity, construct validity, concurrent validity, internal consistency reliability, and test-retest reliability. Results Exploratory factor analysis and confirmatory factor analysis demonstrated that the 18-item scale comprised five factors (functional independence, symptom recognition and management, positive mental health, participation and support in social activities, and a safe environment). The results of EFA showed that the factor loadings ranging from 0.51 to 0.87. The results of CFA were χ2=375.83, χ2/df=2.14, RMSEA 0.07, SRMR 0.05, GFI 0.88, TLI 0.91, and CFI 0.92 for the 18-item scale. The reliabilities of the scale were 0.91 for Cronbach’s alpha and 0.82 for test-retest reliability. Conclusions The HFS-SC has acceptable validity and reliability and is expected to be useful for evaluating the levels of self-care for older adults undergoing hip fracture surgery and adjusting at the post-fracture period at the community or out-patient department. Supplementary Information The online version contains supplementary material available at 10.1186/s12912-022-00982-3.

Project description:BackgroundFractures as a result of osteoporosis and low bone mass are common and give rise to significant clinical, personal, and economic burden. Even after a fracture occurs, high fracture risk remains widely underdiagnosed and undertreated. Common fracture risk assessment tools utilize a subset of clinical risk factors for prediction, and often require manual data entry. Furthermore, these tools predict risk over the long term and do not explicitly provide short-term risk estimates necessary to identify patients likely to experience a fracture in the next 1-2 years.ObjectiveThe goal of this study was to develop and evaluate an algorithm for the identification of patients at risk of fracture in a subsequent 1- to 2-year period. In order to address the aforementioned limitations of current prediction tools, this approach focused on a short-term timeframe, automated data entry, and the use of longitudinal data to inform the predictions.MethodsUsing retrospective electronic health record data from over 1,000,000 patients, we developed Crystal Bone, an algorithm that applies machine learning techniques from natural language processing to the temporal nature of patient histories to generate short-term fracture risk predictions. Similar to how language models predict the next word in a given sentence or the topic of a document, Crystal Bone predicts whether a patient's future trajectory might contain a fracture event, or whether the signature of the patient's journey is similar to that of a typical future fracture patient. A holdout set with 192,590 patients was used to validate accuracy. Experimental baseline models and human-level performance were used for comparison.ResultsThe model accurately predicted 1- to 2-year fracture risk for patients aged over 50 years (area under the receiver operating characteristics curve [AUROC] 0.81). These algorithms outperformed the experimental baselines (AUROC 0.67) and showed meaningful improvements when compared to retrospective approximation of human-level performance by correctly identifying 9649 of 13,765 (70%) at-risk patients who did not receive any preventative bone-health-related medical interventions from their physicians.ConclusionsThese findings indicate that it is possible to use a patient's unique medical history as it changes over time to predict the risk of short-term fracture. Validating and applying such a tool within the health care system could enable automated and widespread prediction of this risk and may help with identification of patients at very high risk of fracture.

Dataset Information

A Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography: System Development and Validation Study.

Background

Objective

Methods

Results

Conclusions

Publications

A Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography: System Development and Validation Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets