Dataset Information

Accuracy of paper-and-pencil systematic observation versus computer-aided systems.

ABSTRACT: Computer-aided behavior observation is gradually supplanting paper-and-pencil approaches to behavior observation, but there is a dearth of evidence on the relative accuracy of paper-and-pencil versus computer-aided behavior observation formats in the literature. The current study evaluated the accuracy resulting from paper-and-pencil observation and from two computer-aided behavior observation methods: The Observer XT® desktop software and the Big Eye Observer® smartphone application. Twelve postgraduate students without behavior observation experience underwent a behavior observation training protocol. As part of a multi-element design, participants recorded 60 real clinical sessions randomly assigned to one of the three observation methods. All three methods produced high levels of accuracy (paper-and-pencil, .88 ± .01; The Observer XT, .84 ± .01; Big Eye Observer, .84 ± .01). A mixed linear model analysis indicated that paper-and-pencil observation produced marginally superior accuracy values, whereas the accuracy produced by The Observer XT and Big Eye Observer did not differ. The analysis suggests that accuracy of recording was mediated by the number of recordable events in the observation videos. The implications of these findings for research and practice are discussed.

SUBMITTER: Virues-Ortega J

PROVIDER: S-EPMC10027644 | biostudies-literature | 2023 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Accuracy of paper-and-pencil systematic observation versus computer-aided systems.

Virues-Ortega Javier J Casas Candida Delgado CD Martin Neil N Tarifa-Rodriguez Aida A Hidalgo Antonio Jesús Reina AJR Cox Alison D AD Navarro Guzmán José I JI

Behavior research methods 20220427 2

Computer-aided behavior observation is gradually supplanting paper-and-pencil approaches to behavior observation, but there is a dearth of evidence on the relative accuracy of paper-and-pencil versus computer-aided behavior observation formats in the literature. The current study evaluated the accuracy resulting from paper-and-pencil observation and from two computer-aided behavior observation methods: The Observer XT® desktop software and the Big Eye Observer® smartphone application. Twelve pos ...[more]

PMID: 35476296

Similar Datasets

Project description:Importance:The recent advances in the field of machine learning have raised expectations that computer-aided diagnosis will become the standard for the diagnosis of melanoma. Objective:To critically review the current literature and compare the diagnostic accuracy of computer-aided diagnosis with that of human experts. Data Sources:The MEDLINE, arXiv, and PubMed Central databases were searched to identify eligible studies published between January 1, 2002, and December 31, 2018. Study Selection:Studies that reported on the accuracy of automated systems for melanoma were selected. Search terms included melanoma, diagnosis, detection, computer aided, and artificial intelligence. Data Extraction and Synthesis:Evaluation of the risk of bias was performed using the QUADAS-2 tool, and quality assessment was based on predefined criteria. Data were analyzed from February 1 to March 10, 2019. Main Outcomes and Measures:Summary estimates of sensitivity and specificity and summary receiver operating characteristic curves were the primary outcomes. Results:The literature search yielded 1694 potentially eligible studies, of which 132 were included and 70 offered sufficient information for a quantitative analysis. Most studies came from the field of computer science. Prospective clinical studies were rare. Combining the results for automated systems gave a melanoma sensitivity of 0.74 (95% CI, 0.66-0.80) and a specificity of 0.84 (95% CI, 0.79-0.88). Sensitivity was lower in studies that used independent test sets than in those that did not (0.51; 95% CI, 0.34-0.69 vs 0.82; 95% CI, 0.77-0.86; P?<?.001); however, the specificity was similar (0.83; 95% CI, 0.71-0.91 vs 0.85; 95% CI, 0.80-0.88; P?=?.67). In comparison with dermatologists' diagnosis, computer-aided diagnosis showed similar sensitivities and a 10 percentage points lower specificity, but the difference was not statistically significant. Studies were heterogeneous and substantial risk of bias was found in all but 4 of the 70 studies included in the quantitative analysis. Conclusions and Relevance:Although the accuracy of computer-aided diagnosis for melanoma detection is comparable to that of experts, the real-world applicability of these systems is unknown and potentially limited owing to overfitting and the risk of bias of the studies at hand.

Project description:PurposeThe present systematic review aimed to investigate the accuracy of zygomatic implant (ZI) placement using dynamic computer-aided surgery (d-CAIS), static computer-aided surgery (s-CAIS), and a free-hand approach in patients with severe atrophic edentulous maxilla and/or deficient maxilla.MethodsElectronic and manual literature searches until May 2023 were performed in the PubMed/Medline, Scopus, Cochrane Library, and Web of Science databases. Clinical trials and cadaver studies were selected. The primary outcome was planned/placed deviation. Secondary outcomes were to evaluate the survival of ZI and surgical complications. Random-effects meta-analyses were conducted and meta-regression was utilized to compare fiducial registration amounts for d-CAIS and the different designs of s-CAIS.ResultsA total of 14 studies with 511 ZIs were included (Nobel Biocare: 274, Southern Implant: 42, SIN Implant: 16, non-mentioned: 179). The pooled mean ZI deviations from the d-CAIS group were 1.81 mm (95% CI: 1.34-2.29) at the entry point and 2.95 mm (95% CI: 1.66-4.24) at the apex point, and angular deviations were 3.49 degrees (95% CI: 2.04-4.93). The pooled mean ZI deviations from the s-CAIS group were 1.19 mm (95% CI: 0.83-1.54) at the entry point and 1.80 mm (95% CI: 1.10-2.50) at the apex point, and angular deviations were 2.15 degrees (95% CI: 1.43-2.88). The pooled mean ZI deviations from the free-hand group were 2.04 mm (95% CI: 1.69-2.39) at the entry point and 3.23 mm (95% CI: 2.34-4.12) at the apex point, and angular deviations were 4.92 degrees (95% CI: 3.86-5.98). There was strong evidence of differences in the average entry, apex, and angular deviation between the navigation, surgical guide, and free-hand groups (p < 0.01). A significant inverse correlation was observed between the number of fiducial screws and the planned/placed deviation regarding entry, apex, and angular measurements.ConclusionUsing d-CAIS and modified s-CAIS for ZI surgery has shown clinically acceptable outcomes regarding average entry, apex, and angular deviations. The maximal deviation values were predominantly observed in the conventional s-CAIS. Surgeons should be mindful of potential deviations and complications regardless of the decision making in different guide approaches.

Project description:After the US Food and Drug Administration (FDA) approved computer-aided detection (CAD) for mammography in 1998, and the Centers for Medicare and Medicaid Services (CMS) provided increased payment in 2002, CAD technology disseminated rapidly. Despite sparse evidence that CAD improves accuracy of mammographic interpretations and costs over $400 million a year, CAD is currently used for most screening mammograms in the United States.To measure performance of digital screening mammography with and without CAD in US community practice.We compared the accuracy of digital screening mammography interpreted with (n = 495 818) vs without (n = 129 807) CAD from 2003 through 2009 in 323 973 women. Mammograms were interpreted by 271 radiologists from 66 facilities in the Breast Cancer Surveillance Consortium. Linkage with tumor registries identified 3159 breast cancers in 323 973 women within 1 year of the screening.Mammography performance (sensitivity, specificity, and screen-detected and interval cancers per 1000 women) was modeled using logistic regression with radiologist-specific random effects to account for correlation among examinations interpreted by the same radiologist, adjusting for patient age, race/ethnicity, time since prior mammogram, examination year, and registry. Conditional logistic regression was used to compare performance among 107 radiologists who interpreted mammograms both with and without CAD.Screening performance was not improved with CAD on any metric assessed. Mammography sensitivity was 85.3% (95% CI, 83.6%-86.9%) with and 87.3% (95% CI, 84.5%-89.7%) without CAD. Specificity was 91.6% (95% CI, 91.0%-92.2%) with and 91.4% (95% CI, 90.6%-92.0%) without CAD. There was no difference in cancer detection rate (4.1 in 1000 women screened with and without CAD). Computer-aided detection did not improve intraradiologist performance. Sensitivity was significantly decreased for mammograms interpreted with vs without CAD in the subset of radiologists who interpreted both with and without CAD (odds ratio, 0.53; 95% CI, 0.29-0.97).Computer-aided detection does not improve diagnostic accuracy of mammography. These results suggest that insurers pay more for CAD with no established benefit to women.

Dataset Information

Accuracy of paper-and-pencil systematic observation versus computer-aided systems.

Publications

Accuracy of paper-and-pencil systematic observation versus computer-aided systems.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets