Is Deep Learning On Par with Human Observers for Detection of Radiographically Visible and Occult Fractures of the Scaphoid?
Ontology highlight
ABSTRACT:
SUBMITTER: Langerhuizen DWG
PROVIDER: S-EPMC7571968 | biostudies-literature | 2020 Nov
REPOSITORIES: biostudies-literature
Publications
Is Deep Learning On Par with Human Observers for Detection of Radiographically Visible and Occult Fractures of the Scaphoid?
Langerhuizen David W G DWG Bulstra Anne Eva J AEJ Janssen Stein J SJ Ring David D Kerkhoffs Gino M M J GMMJ Jaarsma Ruurd L RL Doornberg Job N JN
Clinical orthopaedics and related research 20201101 11
<h4>Background</h4>Preliminary experience suggests that deep learning algorithms are nearly as good as humans in detecting common, displaced, and relatively obvious fractures (such as, distal radius or hip fractures). However, it is not known whether this also is true for subtle or relatively nondisplaced fractures that are often difficult to see on radiographs, such as scaphoid fractures.<h4>Questions/purposes</h4>(1) What is the diagnostic accuracy, sensitivity, and specificity of a deep learn ...[more]
Similar Datasets
Project description:ObjectivesTo assess how an artificial intelligence (AI) algorithm performs against five experienced musculoskeletal radiologists in diagnosing scaphoid fractures and whether it aids their diagnosis on conventional multi-view radiographs.MethodsFour datasets of conventional hand, wrist, and scaphoid radiographs were retrospectively acquired at two hospitals (hospitals A and B). Dataset 1 (12,990 radiographs from 3353 patients, hospital A) and dataset 2 (1117 radiographs from 394 patients, hospital B) were used for training and testing a scaphoid localization and laterality classification component. Dataset 3 (4316 radiographs from 840 patients, hospital A) and dataset 4 (688 radiographs from 209 patients, hospital B) were used for training and testing the fracture detector. The algorithm was compared with the radiologists in an observer study. Evaluation metrics included sensitivity, specificity, positive predictive value (PPV), area under the characteristic operating curve (AUC), Cohen's kappa coefficient (κ), fracture localization precision, and reading time.ResultsThe algorithm detected scaphoid fractures with a sensitivity of 72%, specificity of 93%, PPV of 81%, and AUC of 0.88. The AUC of the algorithm did not differ from each radiologist (0.87 [radiologists' mean], p ≥ .05). AI assistance improved five out of ten pairs of inter-observer Cohen's κ agreements (p < .05) and reduced reading time in four radiologists (p < .001), but did not improve other metrics in the majority of radiologists (p ≥ .05).ConclusionsThe AI algorithm detects scaphoid fractures on conventional multi-view radiographs at the level of five experienced musculoskeletal radiologists and could significantly shorten their reading time.Key points• An artificial intelligence algorithm automatically detects scaphoid fractures on conventional multi-view radiographs at the same level of five experienced musculoskeletal radiologists. • There is preliminary evidence that automated scaphoid fracture detection can significantly shorten the reading time of musculoskeletal radiologists.
| S-EPMC9935716 | biostudies-literature
Project description:ImportanceScaphoid fractures are the most common carpal fracture, but as many as 20% are not visible (ie, occult) in the initial injury radiograph; untreated scaphoid fractures can lead to degenerative wrist arthritis and debilitating pain, detrimentally affecting productivity and quality of life. Occult scaphoid fractures are among the primary causes of scaphoid nonunions, secondary to delayed diagnosis.ObjectiveTo develop and validate a deep convolutional neural network (DCNN) that can reliably detect both apparent and occult scaphoid fractures from radiographic images.Design, setting, and participantsThis diagnostic study used a radiographic data set compiled for all patients presenting to Chang Gung Memorial Hospital (Taipei, Taiwan) and Michigan Medicine (Ann Arbor) with possible scaphoid fractures between January 2001 and December 2019. This group was randomly split into training, validation, and test data sets. The images were passed through a detection model to crop around the scaphoid and were then used to train a DCNN model based on the EfficientNetB3 architecture to classify apparent and occult scaphoid fractures. Data analysis was conducted from January to October 2020.ExposuresA DCNN trained to discriminate radiographs with normal and fractured scaphoids.Main outcomes and measuresArea under the receiver operating characteristic curve (AUROC), sensitivity, and specificity. Fracture localization was assessed using gradient-weighted class activation mapping.ResultsOf the 11 838 included radiographs (4917 [41.5%] with scaphoid fracture; 6921 [58.5%] without scaphoid fracture), 8356 (70.6%) were used for training, 1177 (9.9%) for validation, and 2305 (19.5%) for testing. In the testing test, the first DCNN achieved an overall sensitivity and specificity of 87.1% (95% CI, 84.8%-89.2%) and 92.1% (95% CI, 90.6%-93.5%), respectively, with an AUROC of 0.955 in distinguishing scaphoid fractures from scaphoids without fracture. Gradient-weighted class activation mapping closely corresponded to visible fracture sites. The second DCNN achieved an overall sensitivity of 79.0% (95% CI, 70.6%-71.6%) and specificity of 71.6% (95% CI, 69.0%-74.1%) with an AUROC of 0.810 when examining negative cases from the first model. Two-stage examination identified 20 of 22 cases (90.9%) of occult fracture.Conclusions and relevanceIn this study, DCNN models were trained to identify scaphoid fractures. This suggests that such models may be able to assist with radiographic detection of occult scaphoid fractures that are not visible to human observers and to reliably detect fractures of other small bones.
| S-EPMC8103226 | biostudies-literature
Project description:Gastric adenocarcinoma is an aggressive cancer that demonstrates heterogeneous biology depending on patient ethnicity, tumor location, tumor type, and genetic profile. It remains the third leading cause of cancer deaths worldwide and was estimated to result in 782,000 deaths in 2018. Challenges exist in accurately assessing the disease burden, as available radiological staging often underestimates metastatic disease. This diagnostic handicap, along with the poor understanding of the heterogeneous biology of gastric cancer, has hindered the development of effective therapeutic solutions and thus halted improvement in patient outcomes over the last few decades. The management of occult peritoneal disease is complicated, as most patients are understaged by standard imaging studies and therefore thought to have local diseases. In this article, we systematically review recent literature on the limitations that are associated with standard radiographic staging, discuss recent molecular biology advances to better identify and diagnose occult peritoneal disease, and propose possible management strategies to approach this complicated clinical problem.
| S-EPMC7139817 | biostudies-literature
Project description:Atrial fibrillation (AF) often escapes detection, given its frequent paroxysmal and asymptomatic presentation. Deep learning of transthoracic echocardiograms (TTEs), which have structural information, could help identify occult AF. We created a two-stage deep learning algorithm using a video-based convolutional neural network model that (1) distinguished whether TTEs were in sinus rhythm or AF and then (2) predicted which of the TTEs in sinus rhythm were in patients who had experienced AF within 90 days. Our model, trained on 111,319 TTE videos, distinguished TTEs in AF from those in sinus rhythm with high accuracy in a held-out test cohort (AUC 0.96 (0.95-0.96), AUPRC 0.91 (0.90-0.92)). Among TTEs in sinus rhythm, the model predicted the presence of concurrent paroxysmal AF (AUC 0.74 (0.71-0.77), AUPRC 0.19 (0.16-0.23)). Model discrimination remained similar in an external cohort of 10,203 TTEs (AUC of 0.69 (0.67-0.70), AUPRC 0.34 (0.31-0.36)). Performance held across patients who were women (AUC 0.76 (0.72-0.81)), older than 65 years (0.73 (0.69-0.76)), or had a CHA2DS2VASc ≥2 (0.73 (0.79-0.77)). The model performed better than using clinical risk factors (AUC 0.64 (0.62-0.67)), TTE measurements (0.64 (0.62-0.67)), left atrial size (0.63 (0.62-0.64)), or CHA2DS2VASc (0.61 (0.60-0.62)). An ensemble model in a cohort subset combining the TTE model with an electrocardiogram (ECGs) deep learning model performed better than using the ECG model alone (AUC 0.81 vs. 0.79, p = 0.01). Deep learning using TTEs can predict patients with active or occult AF and could be used for opportunistic AF screening that could lead to earlier treatment.
| S-EPMC11016113 | biostudies-literature
Project description:IntroductionThe aim of this study was to develop and validate an easy to use clinical decision rule, applicable in the ED that limits the number of unnecessary cast immobilizations and diagnostic follow-up in suspected scaphoid injury, without increasing the risk of missing fractures.MethodsA prospective multicenter study was conducted that consisted of three components: (1) derivation of a clinical prediction model for detecting scaphoid fractures in adult patients following wrist trauma; (2) internal validation of the model; (3) design of a clinical decision rule. The predictors used were: sex, age, swelling of the anatomic snuffbox, tenderness in the anatomic snuffbox, scaphoid tubercle tenderness, painful ulnar deviation and painful axial thumb compression. The outcome measure was the presence of a scaphoid fracture, diagnosed on either initial radiographs or during re-evaluation after 1-2 weeks or on additional imaging (radiographs/MRI/CT). After multivariate logistic regression analysis and bootstrapping, the regression coefficient for each significant predictor was calculated. The effect of the rule was determined by calculating the number of missed scaphoid fractures and reduction of suspected fractures that required a cast.ResultsA consecutive series of 893 patients with acute wrist injury was included. Sixty-eight patients (7.6%) were diagnosed with a scaphoid fracture. The final prediction rule incorporated sex, swelling of the anatomic snuffbox, tenderness in the anatomic snuffbox, painful ulnar deviation and painful axial thumb compression. Internal validation of the prediction rule showed a sensitivity of 97% and a specificity of 20%. Using this rule, a 15% reduction in unnecessary immobilization and imaging could be achieved with a 50% decreased risk of missing a fracture compared with current clinical practice.ConclusionsThis dataset provided a simple clinical decision rule for scaphoid fractures following acute wrist injury that limits unnecessary immobilization and imaging with a decreased risk of missing a fracture compared to current clinical practice.Clinical prediction rule1/(1 + EXP (-(0.649662618 × if man) + (0.51353467826 × if swelling anatomic snuffbox) + (-0.79038263985 × if painful palpation anatomic snuffbox) + (0.57681198857 × if painful ulnar deviation) + (0.66499549728 × if painful thumb compression)-1.685).Trial registrationTrial register NTR 2544, www.trialregister.nl.
| S-EPMC7109163 | biostudies-literature
Project description:ImportanceOccult peritoneal metastasis frequently occurs in patients with advanced gastric cancer and is poorly diagnosed with currently available tools. Because the presence of peritoneal metastasis precludes the possibility of curative surgery, there is an unmet need for a noninvasive approach to reliably identify patients with occult peritoneal metastasis.ObjectiveTo assess the use of a deep learning model for predicting occult peritoneal metastasis based on preoperative computed tomography images.Design, setting, and participantsIn this multicenter, retrospective cohort study, a deep convolutional neural network, the Peritoneal Metastasis Network (PMetNet), was trained to predict occult peritoneal metastasis based on preoperative computed tomography images. Data from a cohort of 1225 patients with gastric cancer who underwent surgery at Sun Yat-sen University Cancer Center (Guangzhou, China) were used for training purposes. To externally validate the model, data were collected from 2 independent cohorts comprising a total of 753 patients with gastric cancer who underwent surgery at Nanfang Hospital (Guangzhou, China) or the Third Affiliated Hospital of Southern Medical University (Guangzhou, China). The status of peritoneal metastasis for all patients was confirmed by pathological examination of pleural specimens obtained during surgery. Detailed clinicopathological data were collected for each patient. Data analysis was performed between September 1, 2019, and January 31, 2020.Main outcomes and measuresThe area under the receiver operating characteristic curve (AUC) and decision curve were analyzed to evaluate performance in predicting occult peritoneal metastasis.ResultsA total of 1978 patients (mean [SD] age, 56.0 [12.2] years; 1350 [68.3%] male) were included in the study. The PMetNet model achieved an AUC of 0.946 (95% CI, 0.927-0.965), with a sensitivity of 75.4% and a specificity of 92.9% in external validation cohort 1. In external validation cohort 2, the AUC was 0.920 (95% CI, 0.848-0.992), with a sensitivity of 87.5% and a specificity of 98.2%. The discrimination performance of PMetNet was substantially higher than conventional clinicopathological factors (AUC range, 0.51-0.63). In multivariable logistic regression analysis, PMetNet was an independent predictor of occult peritoneal metastasis.Conclusions and relevanceThe findings of this cohort study suggest that the PMetNet model can serve as a reliable noninvasive tool for early identification of patients with clinically occult peritoneal metastasis, which will inform individualized preoperative treatment decision-making and may avoid unnecessary surgery and complications. These results warrant further validation in prospective studies.
| S-EPMC7786251 | biostudies-literature
Project description:BackgroundDiagnosis of rib fractures plays an important role in identifying trauma severity. However, quickly and precisely identifying the rib fractures in a large number of CT images with increasing number of patients is a tough task, which is also subject to the qualification of radiologist. We aim at a clinically applicable automatic system for rib fracture detection and segmentation from CT scans.MethodsA total of 7,473 annotated traumatic rib fractures from 900 patients in a single center were enrolled into our dataset, named RibFrac Dataset, which were annotated with a human-in-the-loop labeling procedure. We developed a deep learning model, named FracNet, to detect and segment rib fractures. 720, 60 and 120 patients were randomly split as training cohort, tuning cohort and test cohort, respectively. Free-Response ROC (FROC) analysis was used to evaluate the sensitivity and false positives of the detection performance, and Intersection-over-Union (IoU) and Dice Coefficient (Dice) were used to evaluate the segmentation performance of predicted rib fractures. Observer studies, including independent human-only study and human-collaboration study, were used to benchmark the FracNet with human performance and evaluate its clinical applicability. A annotated subset of RibFrac Dataset, including 420 for training, 60 for tuning and 120 for test, as well as our code for model training and evaluation, was open to research community to facilitate both clinical and engineering research.FindingsOur method achieved a detection sensitivity of 92.9% with 5.27 false positives per scan and a segmentation Dice of 71.5%on the test cohort. Human experts achieved much lower false positives per scan, while underperforming the deep neural networks in terms of detection sensitivities with longer time in diagnosis. With human-computer collobration, human experts achieved higher detection sensitivities than human-only or computer-only diagnosis.InterpretationThe proposed FracNet provided increasing detection sensitivity of rib fractures with significantly decreased clinical time consumed, which established a clinically applicable method to assist the radiologist in clinical practice.FundingA full list of funding bodies that contributed to this study can be found in the Acknowledgements section. The funding sources played no role in the study design; collection, analysis, and interpretation of data; writing of the report; or decision to submit the article for publication .
| S-EPMC7670192 | biostudies-literature
Project description:ObjectiveTo assess the diagnostic performance of a deep learning-based algorithm for automated detection of acute and chronic rib fractures on whole-body trauma CT.Materials and methodsWe retrospectively identified all whole-body trauma CT scans referred from the emergency department of our hospital from January to December 2018 (n = 511). Scans were categorized as positive (n = 159) or negative (n = 352) for rib fractures according to the clinically approved written CT reports, which served as the index test. The bone kernel series (1.5-mm slice thickness) served as an input for a detection prototype algorithm trained to detect both acute and chronic rib fractures based on a deep convolutional neural network. It had previously been trained on an independent sample from eight other institutions (n = 11455).ResultsAll CTs except one were successfully processed (510/511). The algorithm achieved a sensitivity of 87.4% and specificity of 91.5% on a per-examination level [per CT scan: rib fracture(s): yes/no]. There were 0.16 false-positives per examination (= 81/510). On a per-finding level, there were 587 true-positive findings (sensitivity: 65.7%) and 307 false-negatives. Furthermore, 97 true rib fractures were detected that were not mentioned in the written CT reports. A major factor associated with correct detection was displacement.ConclusionWe found good performance of a deep learning-based prototype algorithm detecting rib fractures on trauma CT on a per-examination level at a low rate of false-positives per case. A potential area for clinical application is its use as a screening tool to avoid false-negative radiology reports.
| S-EPMC7289702 | biostudies-literature
Project description:Background and purposeArtificial intelligence decision support systems are a rapidly growing class of tools to help manage ever-increasing imaging volumes. The aim of this study was to evaluate the performance of an artificial intelligence decision support system, Aidoc, for the detection of cervical spinal fractures on noncontrast cervical spine CT scans and to conduct a failure mode analysis to identify areas of poor performance.Materials and methodsThis retrospective study included 1904 emergent noncontrast cervical spine CT scans of adult patients (60 [SD, 22] years, 50.3% men). The presence of cervical spinal fracture was determined by Aidoc and an attending neuroradiologist; discrepancies were independently adjudicated. Algorithm performance was assessed by calculation of the diagnostic accuracy, and a failure mode analysis was performed.ResultsAidoc and the neuroradiologist's interpretation were concordant in 91.5% of cases. Aidoc correctly identified 67 of 122 fractures (54.9%) with 106 false-positive flagged studies. Diagnostic performance was calculated as the following: sensitivity, 54.9% (95% CI, 45.7%-63.9%); specificity, 94.1% (95% CI, 92.9%-95.1%); positive predictive value, 38.7% (95% CI, 33.1%-44.7%); and negative predictive value, 96.8% (95% CI, 96.2%-97.4%). Worsened performance was observed in the detection of chronic fractures; differences in diagnostic performance were not altered by study indication or patient characteristics.ConclusionsWe observed poor diagnostic accuracy of an artificial intelligence decision support system for the detection of cervical spine fractures. Many similar algorithms have also received little or no external validation, and this study raises concerns about their generalizability, utility, and rapid pace of deployment. Further rigorous evaluations are needed to understand the weaknesses of these tools before widespread implementation.
| S-EPMC8367597 | biostudies-literature
Project description:Many previous studies, including the Next Generation Sequencing (NGS)-based ones, have shown the critical roles of RNA editing in biomedicine. Direct RNA sequencing emerges as another powerful technique to advance the understanding of RNA editing by new paradigms, especially in single-molecule and long-range characterization. The urgent gap is the accurate and robust identification of RNA editing at the single-molecule and single-nucleotide resolution from direct RNA sequencing. This is challenging due to the inherent nature of the context-dependence on the raw signals, which requires enormous training data with considerable diversity. Here we propose two coupled measures to address them: 1) an abductive deep learning strategy implemented as the software ReDD fully utilizes the widely accessible NGS-based RNA editing data as indirect labels of direct RNA sequencing to achieve the detection at the single-molecule level; 2) a cloud-based platform Argo-ReDD serves as a central database for assembling large and diverse data from the community to continuously train the abductive deep learning model, which also meets the community demand of a user-friendly way to perform RNA editing analyses, such as co-occurrence analysis, quantitative analysis and gene isoform-resolved analysis, based on the specific information from direct RNA sequencing.
2024-05-08 | GSE267030 | GEO