Study on the Reliability and Accuracy of Scolioscope, a New Digital Scoliometer.
Ontology highlight
ABSTRACT:
SUBMITTER: Krekoukias G
PROVIDER: S-EPMC8774419 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:PurposeCrowdsourcing is based on outsourcing computationally intensive tasks to numerous individuals in the online community who have no formal training. Our aim was to develop a novel online tool designed to facilitate large-scale annotation of digital retinal images, and to assess the accuracy of crowdsource grading using this tool, comparing it to expert classification.MethodsWe used 100 retinal fundus photograph images with predetermined disease criteria selected by two experts from a large cohort study. The Amazon Mechanical Turk Web platform was used to drive traffic to our site so anonymous workers could perform a classification and annotation task of the fundus photographs in our dataset after a short training exercise. Three groups were assessed: masters only, nonmasters only and nonmasters with compulsory training. We calculated the sensitivity, specificity, and area under the curve (AUC) of receiver operating characteristic (ROC) plots for all classifications compared to expert grading, and used the Dice coefficient and consensus threshold to assess annotation accuracy.ResultsIn total, we received 5389 annotations for 84 images (excluding 16 training images) in 2 weeks. A specificity and sensitivity of 71% (95% confidence interval [CI], 69%-74%) and 87% (95% CI, 86%-88%) was achieved for all classifications. The AUC in this study for all classifications combined was 0.93 (95% CI, 0.91-0.96). For image annotation, a maximal Dice coefficient (∼0.6) was achieved with a consensus threshold of 0.25.ConclusionsThis study supports the hypothesis that annotation of abnormalities in retinal images by ophthalmologically naive individuals is comparable to expert annotation. The highest AUC and agreement with expert annotation was achieved in the nonmasters with compulsory training group.Translational relevanceThe use of crowdsourcing as a technique for retinal image analysis may be comparable to expert graders and has the potential to deliver timely, accurate, and cost-effective image analysis.
| S-EPMC5032847 | biostudies-literature
Project description:BACKGROUND:An apical shift in the position of the gingiva beyond the cemento-enamel junction leads to gingival recession. This study aimed to evaluate the reproducibility of digital measurements of gingival recession when compared to conventional measurements taken clinically using periodontal probes. METHODS:Gingival recession was measured at 97 sites in the oral cavity by four examiners using the following methods: CP, direct measurement of gingival recession using William's periodontal probe intraorally; CC, measurements on cast models using a caliper; DP, digital measurement on virtual models obtained by intraoral scanning, and DC, digital measurements on virtual models of dental casts. Intra-class and inter-rater correlations were analyzed. Bland Altman plots were drawn to visually determine the magnitude of differences in any given pair-wise measurements. RESULTS:In this study, good inter-methods reliability was observed for almost all the examiners ranging from 0.907 to 0.918, except for one examiner (0.837). The greatest disagreements between the raters were observed for methods; CP (0.631) followed by CC (0.85), while the best agreements were observed for methods DP (0.9) followed by DC (0.872). CONCLUSION:Variations in measurements between examiners can be reduced by using digital technologies when compared to conventional methods. Improved reproducibility of measurements obtained via intraoral scanning will increase the validity and reliability of future studies that compare different treatment modalities for root coverage.
| S-EPMC6636111 | biostudies-literature
Project description:PurposeTo design a digital phantom data set for computed tomography (CT) perfusion and perfusion-weighted imaging on the basis of the widely accepted tracer kinetic theory in which the true values of cerebral blood flow (CBF), cerebral blood volume (CBV), mean transit time (MTT), and tracer arrival delay are known and to evaluate the accuracy and reliability of postprocessing programs using this digital phantom.Materials and methodsA phantom data set was created by generating concentration-time curves reflecting true values for CBF (2.5-87.5 mL/100 g per minute), CBV (1.0-5.0 mL/100 g), MTT (3.4-24 seconds), and tracer delays (0-3.0 seconds). These curves were embedded in human brain images. The data were analyzed by using 13 algorithms each for CT and magnetic resonance (MR), including five commercial vendors and five academic programs. Accuracy was assessed by using the Pearson correlation coefficient (r) for true values. Delay-, MTT-, or CBV-dependent errors and correlations between time to maximum of residue function (Tmax) were also evaluated.ResultsIn CT, CBV was generally well reproduced (r > 0.9 in 12 algorithms), but not CBF and MTT (r > 0.9 in seven and four algorithms, respectively). In MR, good correlation (r > 0.9) was observed in one-half of commercial programs, while all academic algorithms showed good correlations for all parameters. Most algorithms had delay-dependent errors, especially for commercial software, as well as CBV dependency for CBF or MTT calculation and MTT dependency for CBV calculation. Correlation was good in Tmax except for one algorithm.ConclusionThe digital phantom readily evaluated the accuracy and characteristics of the CT and MR perfusion analysis software. All commercial programs had delay-induced errors and/or insufficient correlations with true values, while academic programs for MR showed good correlations with true values.Supplemental materialhttp://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.12112618/-/DC1.
| S-EPMC3606546 | biostudies-literature
Project description:Digital anthropometry (DA) has been recently developed for body composition evaluation and for postural analysis. The aims of this review are to examine the current state of DA technology, as well as to verify the methods for identifying the best technology to be used in the field of DA by evaluating the reliability and accuracy of the available technologies on the market, and lay the groundwork for future technological developments. A literature search was performed and 28 studies met the inclusion criteria. The reliability and accuracy of DA was high in most studies, especially in the assessment of patients with obesity, although they varied according to the technology used; a good correlation was found between DA and conventional anthropometry (CA) and body composition estimates. DA is less time-consuming and less expensive and could be used as a screening tool before more expensive imaging techniques or as an alternative to other less affordable techniques. At present, DA could be useful in clinical practice, but the heterogeneity of the available studies (different devices used, laser technologies, population examined, etc.) necessitates caution in the interpretation of the obtained results. Furthermore, the need to develop integrated technologies for analyzing body composition according to multi-compartmental models is increasingly evident.
| S-EPMC9864001 | biostudies-literature
Project description:There is significant interest in using neuroimaging data to predict behavior. The predictive models are often interpreted by the computation of feature importance, which quantifies the predictive relevance of an imaging feature. Tian and Zalesky (2021) suggest that feature importance estimates exhibit low split-half reliability, as well as a trade-off between prediction accuracy and feature importance reliability across parcellation resolutions. However, it is unclear whether the trade-off between prediction accuracy and feature importance reliability is universal. Here, we demonstrate that, with a sufficient sample size, feature importance (operationalized as Haufe-transformed weights) can achieve fair to excellent split-half reliability. With a sample size of 2600 participants, Haufe-transformed weights achieve average intra-class correlation coefficients of 0.75, 0.57 and 0.53 for cognitive, personality and mental health measures respectively. Haufe-transformed weights are much more reliable than original regression weights and univariate FC-behavior correlations. Original regression weights are not reliable even with 2600 participants. Intriguingly, feature importance reliability is strongly positively correlated with prediction accuracy across phenotypes. Within a particular behavioral domain, there is no clear relationship between prediction performance and feature importance reliability across regression models. Furthermore, we show mathematically that feature importance reliability is necessary, but not sufficient, for low feature importance error. In the case of linear models, lower feature importance error is mathematically related to lower prediction error. Therefore, higher feature importance reliability might yield lower feature importance error and higher prediction accuracy. Finally, we discuss how our theoretical results relate with the reliability of imaging features and behavioral measures. Overall, the current study provides empirical and theoretical insights into the relationship between prediction accuracy and feature importance reliability.
| S-EPMC10241563 | biostudies-literature
Project description:Use of continuous glucose monitoring (CGM) systems can improve glycemic control, but widespread adoption of CGM utilization has been limited, in part because of real and perceived problems with accuracy and reliability. This study compared accuracy and performance metrics for a new-generation CGM system with those of a previous-generation device.Subjects were enrolled in a 7-day, open-label, multicenter pivotal study. Sensor readings were compared with venous YSI measurements (blood glucose analyzer from YSI Inc., Yellow Springs, OH) every 15 min (±5 min) during in-clinic visits. The aggregate and individual sensor accuracy and reliability of a new CGM system, the Dexcom(®) (San Diego, CA) G4™ PLATINUM (DG4P), were compared with those of the previous CGM system, the Dexcom SEVEN(®) PLUS (DSP).Both study design and subject characteristics were similar. The aggregate mean absolute relative difference (MARD) for DG4P was 13% compared with 16% for DSP (P<0.0001), and 82% of DG4P readings were within ± 20 mg/dL (for YSI ? 80 mg/dL) or 20% of YSI values (for YSI >80 mg/dL) compared with 76% for DSP (P<0.001). Ninety percent of the DG4P sensors had an individual MARD ? 20% compared with only 76% of DSP sensors (P=0.015). Half of DG4P sensors had a MARD less than 12.5% compared with 14% for the DSP sensors (P=0.028). The mean absolute difference for biochemical hypoglycemia (YSI <70 mg/dL) for DG4P was 11 mg/dL compared with 16 mg/dL for DSP (P<0.001).The performance of DG4P was significantly improved compared with that of DSP, which may increase routine clinical use of CGM and improve patient outcomes.
| S-EPMC3781114 | biostudies-literature
Project description:The interpretation of forensic fingerprint evidence relies on the expertise of latent print examiners. The National Research Council of the National Academies and the legal and forensic sciences communities have called for research to measure the accuracy and reliability of latent print examiners' decisions, a challenging and complex problem in need of systematic analysis. Our research is focused on the development of empirical approaches to studying this problem. Here, we report on the first large-scale study of the accuracy and reliability of latent print examiners' decisions, in which 169 latent print examiners each compared approximately 100 pairs of latent and exemplar fingerprints from a pool of 744 pairs. The fingerprints were selected to include a range of attributes and quality encountered in forensic casework, and to be comparable to searches of an automated fingerprint identification system containing more than 58 million subjects. This study evaluated examiners on key decision points in the fingerprint examination process; procedures used operationally include additional safeguards designed to minimize errors. Five examiners made false positive errors for an overall false positive rate of 0.1%. Eighty-five percent of examiners made at least one false negative error for an overall false negative rate of 7.5%. Independent examination of the same comparisons by different participants (analogous to blind verification) was found to detect all false positive errors and the majority of false negative errors in this study. Examiners frequently differed on whether fingerprints were suitable for reaching a conclusion.
| S-EPMC3093498 | biostudies-literature
Project description:The accuracy of static guided implant surgery (sGIS) using conventional planning workflow has been extensively examined; however, more information is required to justify the application of fully digital planning protocol. The purpose of this study was to investigate the clinical accuracy of sGIS with a fully digital planning workflow. Twenty-one partially edentulous patients were enrolled in this prospective study. Cone-beam computed tomography (CBCT) and intraoral scans were taken and superimposed by matching the dental surface images directly (surface registration protocol) or by matching fiducial markers on a stereolithographic (SLA) radiographic template fabricated from the digital data of the intraoral scan (fiducial marker registration protocol). Virtual implant treatment plans were then determined, and tooth-supported SLA surgical guides were fabricated according to the plans. Twenty-six implant surgeries were performed via the surgical guide by one surgeon. Pre- and post-operative CBCT images were superimposed, and the positional and angular deviations between placed and planned implants were measured with metrology software. A total of 43 fully guided implants were placed, in which 25 implants were planned with the surface registration protocol. Implants planned based on the surface registration protocol had a larger mean angular deviation than the fiducial marker registration protocol. No significant differences were found for any deviations of the examined variables. Within the limits of this study, we concluded that the clinical accuracy of the sGIS planned with a fully digital workflow was consistent with the conventional workflow for partially edentulous patients.
| S-EPMC7231012 | biostudies-literature
Project description:In this study, the accuracy (precision and trueness) of digital impressions and the fitness of single crowns manufactured based on digital impressions were evaluated. #14-17 epoxy resin dentitions were made, while full-crown preparations of extracted natural teeth were embedded at #16. (1) To assess precision, deviations among repeated scan models made by intraoral scanner TRIOS and MHT and model scanner D700 and inEos were calculated through best-fit algorithm and three-dimensional (3D) comparison. Root mean square (RMS) and color-coded difference images were offered. (2) To assess trueness, micro computed tomography (micro-CT) was used to get the reference model (REF). Deviations between REF and repeated scan models (from (1)) were calculated. (3) To assess fitness, single crowns were manufactured based on TRIOS, MHT, D700 and inEos scan models. The adhesive gaps were evaluated under stereomicroscope after cross-sectioned. Digital impressions showed lower precision and better trueness. Except for MHT, the means of RMS for precision were lower than 10 μm. Digital impressions showed better internal fitness. Fitness of single crowns based on digital impressions was up to clinical standard. Digital impressions could be an alternative method for single crowns manufacturing.
| S-EPMC5455646 | biostudies-other
Project description:BackgroundComputer-aided diagnosis (CADx) software that provides a second opinion has been widely used to assist physicians with various tasks. In dermatology, however, CADx has been mostly limited to melanoma or melanocytic skin cancer diagnosis. The frequency of non-melanocytic skin cancers and the accessibility of regular digital macrographs have raised interest in developing CADx for broader applications.ObjectivesTo investigate the feasibility of using CADx to diagnose both melanocytic and non-melanocytic skin lesions based on conventional digital photographic images.MethodsThis study was approved by an institutional review board, and the requirement to obtain informed consent was waived. In total, 769 conventional photographs of melanocytic and non-melanocytic skin lesions were retrospectively reviewed and used to develop a CADx system. Conventional and new color-related image features were developed to classify the lesions as benign or malignant using support vector machines (SVMs). The performance of CADx was compared with that of dermatologists.ResultsThe clinicians' overall sensitivity, specificity, and accuracy were 83.33%, 85.88%, and 85.31%, respectively. New color correlation and principal component analysis (PCA) features improved the classification ability of the baseline CADx (p = 0.001). The estimated area under the receiver operating characteristic (ROC) curve (Az) of the proposed CADx system was 0.949, with a sensitivity and specificity of 85.63% and 87.65%, respectively, and a maximum accuracy of 90.64%.ConclusionsWe have developed an effective CADx system to classify both melanocytic and non-melanocytic skin lesions using conventional digital macrographs. The system's performance was similar to that of dermatologists at our institute. Through improved feature extraction and SVM analysis, we found that conventional digital macrographs were feasible for providing useful information for CADx applications. The new color-related features significantly improved CADx applications for skin cancer.
| S-EPMC3817186 | biostudies-literature