Dataset Information

Are Pathologists Self-Aware of Their Diagnostic Accuracy? Metacognition and the Diagnostic Process in Pathology.

ABSTRACT:

Background

Metacognition is a cognitive process that involves self-awareness of thinking, understanding, and performance. This study assesses pathologists' metacognition by examining the association between their diagnostic accuracy and self-reported confidence levels while interpreting skin and breast biopsies.

Design

We studied 187 pathologists from the Melanoma Pathology Study (M-Path) and 115 pathologists from the Breast Pathology Study (B-Path). We measured pathologists' metacognitive ability by examining the area under the curve (AUC), the area under each pathologist's receiver operating characteristic (ROC) curve summarizing the association between confidence and diagnostic accuracy. We investigated possible relationships between this AUC measure, referred to as metacognitive sensitivity, and pathologist attributes. We also assessed whether higher metacognitive sensitivity affected the association between diagnostic accuracy and a secondary diagnostic action such as requesting a second opinion.

Results

We found no significant associations between pathologist clinical attributes and metacognitive AUC. However, we found that pathologists with higher AUC showed a stronger trend to request secondary diagnostic action for inaccurate diagnoses and not for accurate diagnoses compared with pathologists with lower AUC.

Limitations

Pathologists reported confidence in specific diagnostic terms, rather than the broader classes into which the diagnostic terms were later grouped to determine accuracy. In addition, while there is no gold standard for the correct diagnosis to determine the accuracy of pathologists' interpretations, our studies achieved a high-quality reference diagnosis by using the consensus diagnosis of 3 experienced pathologists.

Conclusions

Metacognition can affect clinical decisions. If pathologists have self-awareness that their diagnosis may be inaccurate, they can request additional tests or second opinions, providing the opportunity to correct inaccurate diagnoses.

Highlights

Metacognitive sensitivity varied across pathologists, with most showing higher sensitivity than expected by chance.None of the demographic or clinical characteristics we examined was significantly associated with metacognitive sensitivity.Pathologists with higher metacognitive sensitivity were more likely to request additional tests or second opinions for their inaccurate diagnoses.

SUBMITTER: Clayton DA

PROVIDER: S-EPMC9825636 | biostudies-literature | 2023 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Are Pathologists Self-Aware of Their Diagnostic Accuracy? Metacognition and the Diagnostic Process in Pathology.

Clayton Dayna A DA Eguchi Megan M MM Kerr Kathleen F KF Miyoshi Kiyofumi K Brunyé Tad T TT Drew Trafton T Weaver Donald L DL Elmore Joann G JG

Medical decision making : an international journal of the Society for Medical Decision Making 20220920 2

<h4>Background</h4>Metacognition is a cognitive process that involves self-awareness of thinking, understanding, and performance. This study assesses pathologists' metacognition by examining the association between their diagnostic accuracy and self-reported confidence levels while interpreting skin and breast biopsies.<h4>Design</h4>We studied 187 pathologists from the Melanoma Pathology Study (M-Path) and 115 pathologists from the Breast Pathology Study (B-Path). We measured pathologists' meta ...[more]

PMID: 36124966

Similar Datasets

Project description:CONTEXT:There is increasing interest in using whole slide imaging (WSI) for diagnostic purposes (primary and/or consultation). An important consideration is whether WSI can safely replace conventional light microscopy as the method by which pathologists review histologic sections, cytology slides, and/or hematology slides to render diagnoses. Validation of WSI is crucial to ensure that diagnostic performance based on digitized slides is at least equivalent to that of glass slides and light microscopy. Currently, there are no standard guidelines regarding validation of WSI for diagnostic use. OBJECTIVE:To recommend validation requirements for WSI systems to be used for diagnostic purposes. DESIGN:The College of American Pathologists Pathology and Laboratory Quality Center convened a nonvendor panel from North America with expertise in digital pathology to develop these validation recommendations. A literature review was performed in which 767 international publications that met search term requirements were identified. Studies outside the scope of this effort and those related solely to technical elements, education, and image analysis were excluded. A total of 27 publications were graded and underwent data extraction for evidence evaluation. Recommendations were derived from the strength of evidence determined from 23 of these published studies, open comment feedback, and expert panel consensus. RESULTS:Twelve guideline statements were established to help pathology laboratories validate their own WSI systems intended for clinical use. Validation of the entire WSI system, involving pathologists trained to use the system, should be performed in a manner that emulates the laboratory's actual clinical environment. It is recommended that such a validation study include at least 60 routine cases per application, comparing intraobserver diagnostic concordance between digitized and glass slides viewed at least 2 weeks apart. It is important that the validation process confirm that all material present on a glass slide to be scanned is included in the digital image. CONCLUSIONS:Validation should demonstrate that the WSI system under review produces acceptable digital slides for diagnostic interpretation. The intention of validating WSI systems is to permit the clinical use of this technology in a manner that does not compromise patient care.

Project description:ImportanceA breast pathology diagnosis provides the basis for clinical treatment and management decisions; however, its accuracy is inadequately understood.ObjectivesTo quantify the magnitude of diagnostic disagreement among pathologists compared with a consensus panel reference diagnosis and to evaluate associated patient and pathologist characteristics.Design, setting, and participantsStudy of pathologists who interpret breast biopsies in clinical practices in 8 US states.ExposuresParticipants independently interpreted slides between November 2011 and May 2014 from test sets of 60 breast biopsies (240 total cases, 1 slide per case), including 23 cases of invasive breast cancer, 73 ductal carcinoma in situ (DCIS), 72 with atypical hyperplasia (atypia), and 72 benign cases without atypia. Participants were blinded to the interpretations of other study pathologists and consensus panel members. Among the 3 consensus panel members, unanimous agreement of their independent diagnoses was 75%, and concordance with the consensus-derived reference diagnoses was 90.3%.Main outcomes and measuresThe proportions of diagnoses overinterpreted and underinterpreted relative to the consensus-derived reference diagnoses were assessed.ResultsSixty-five percent of invited, responding pathologists were eligible and consented to participate. Of these, 91% (N = 115) completed the study, providing 6900 individual case diagnoses. Compared with the consensus-derived reference diagnosis, the overall concordance rate of diagnostic interpretations of participating pathologists was 75.3% (95% CI, 73.4%-77.0%; 5194 of 6900 interpretations). Among invasive carcinoma cases (663 interpretations), 96% (95% CI, 94%-97%) were concordant, and 4% (95% CI, 3%-6%) were underinterpreted; among DCIS cases (2097 interpretations), 84% (95% CI, 82%-86%) were concordant, 3% (95% CI, 2%-4%) were overinterpreted, and 13% (95% CI, 12%-15%) were underinterpreted; among atypia cases (2070 interpretations), 48% (95% CI, 44%-52%) were concordant, 17% (95% CI, 15%-21%) were overinterpreted, and 35% (95% CI, 31%-39%) were underinterpreted; and among benign cases without atypia (2070 interpretations), 87% (95% CI, 85%-89%) were concordant and 13% (95% CI, 11%-15%) were overinterpreted. Disagreement with the reference diagnosis was statistically significantly higher among biopsies from women with higher (n = 122) vs lower (n = 118) breast density on prior mammograms (overall concordance rate, 73% [95% CI, 71%-75%] for higher vs 77% [95% CI, 75%-80%] for lower, P < .001), and among pathologists who interpreted lower weekly case volumes (P < .001) or worked in smaller practices (P = .034) or nonacademic settings (P = .007).Conclusions and relevanceIn this study of pathologists, in which diagnostic interpretation was based on a single breast biopsy slide, overall agreement between the individual pathologists' interpretations and the expert consensus-derived reference diagnoses was 75.3%, with the highest level of concordance for invasive carcinoma and lower levels of concordance for DCIS and atypia. Further research is needed to understand the relationship of these findings with patient management.

Project description:BackgroundAlthough pathology informatics (PI) is essential to modern pathology practice, the field is often poorly understood. Pathologists who have received little to no exposure to informatics, either in training or in practice, may not recognize the roles that informatics serves in pathology. The purpose of this study was to characterize perceptions of PI by noninformatics-oriented pathologists and to do so at two large centers with differing informatics environments.MethodsPathology trainees and staff at Cleveland Clinic (CC) and Massachusetts General Hospital (MGH) were surveyed. At MGH, pathology department leadership has promoted a pervasive informatics presence through practice, training, and research. At CC, PI efforts focus on production systems that serve a multi-site integrated health system and a reference laboratory, and on the development of applications oriented to department operations. The survey assessed perceived definition of PI, interest in PI, and perceived utility of PI.ResultsThe survey was completed by 107 noninformatics-oriented pathologists and trainees. A majority viewed informatics positively. Except among MGH trainees, confusion of PI with information technology (IT) and help desk services was prominent, even in those who indicated they understood informatics. Attendings and trainees indicated desire to learn more about PI. While most acknowledged that having some level of PI knowledge would be professionally useful and advantageous, only a minority plan to utilize it.ConclusionsInformatics is viewed positively by the majority of noninformatics pathologists at two large centers with differing informatics orientations. Differences in departmental informatics culture can be attributed to the varying perceptions of PI by different individuals. Incorrect perceptions exist, such as conflating PI with IT and help desk services, even among those who claim to understand PI. Further efforts by the PI community could address such misperceptions, which could help enable a better understanding of what PI is and is not, and potentially lead to increased acceptance by non-informaticist pathologists.

Project description:ImportancePhosphorylated tau (pTau) is a specific blood biomarker for Alzheimer's disease (AD) pathology, with pTau217 considered to have the most utility. However, availability of pTau217 tests for research and clinical use has been limited. Expanding access to this highly accurate AD biomarker is crucial for wider evaluation and implementation of AD blood tests.ObjectiveTo determine the utility of a novel and commercially available Single molecule array (Simoa) for plasma pTau217 (ALZpath) to detect AD pathology. To evaluate references ranges for abnormal Aβ across three selected cohorts.Design setting participantsThree single-centre observational cohorts were involved in the study: Translational Biomarkers in Aging and Dementia (TRIAD), Wisconsin Registry for Alzheimer's Prevention (WRAP), and Sant Pau Initiative on Neurodegeneration (SPIN). MRI, Aβ-PET, and tau-PET data were available for TRIAD and WRAP, while CSF biomarkers were additionally measured in a subset of TRIAD and SPIN. Plasma measurements of pTau181, pTau217 (ALZpath), pTau231, Aβ42/40, GFAP, and NfL, were available for all cohorts. Longitudinal blood biomarker data spanning 3 years for TRIAD and 8 years for WRAP were included.ExposuresMRI, Aβ-PET, tau-PET, CSF biomarkers (Aβ42/40 and pTau immunoassays) and plasma pTau217 (ALZpath Simoa).Main outcomes and measuresThe accuracy of plasma pTau217 for detecting abnormal amyloid and tau pathology. Longitudinal pTau217 change according to baseline pathology status.ResultsThe study included 786 participants (mean [SD] age, 66.3 [9.7] years; 504 females [64.1%]) were included in the study. High accuracy was observed in identifying elevated Aβ (AUC, 0.92-0.96; 95%CI 0.89-0.99) and tau pathology (AUC, 0.93-0.97; 95%CI 0.84-0.99) across all cohorts. These accuracies were significantly higher than other plasma biomarker combinations and comparable to CSF biomarkers. The detection of abnormal Aβ pathology using binary or three-range references yielded reproducible results. Longitudinally, plasma pTau217 showed an annual increase only in Aβ-positive individuals, with the highest increase observed in those with tau-positivity.Conclusions and relevanceThe ALZpath plasma pTau217 Simoa assay accurately identifies biological AD, comparable to CSF biomarkers, with reproducible cut-offs across cohorts. It detects longitudinal changes, including at the preclinical stage, and is the first widely available, accessible, and scalable blood test for pTau217 detection.

Dataset Information

Are Pathologists Self-Aware of Their Diagnostic Accuracy? Metacognition and the Diagnostic Process in Pathology.

Background

Design

Results

Limitations

Conclusions

Highlights

Publications

Are Pathologists Self-Aware of Their Diagnostic Accuracy? Metacognition and the Diagnostic Process in Pathology.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets