Dataset Information

Selection with Variation in Diagnostic Skill: Evidence from Radiologists.

ABSTRACT: Physicians, judges, teachers, and agents in many other settings differ systematically in the decisions they make when faced with similar cases. Standard approaches to interpreting and exploiting such differences assume they arise solely from variation in preferences. We develop an alternative framework that allows variation in preferences and diagnostic skill and show that both dimensions may be partially identified in standard settings under quasi-random assignment. We apply this framework to study pneumonia diagnoses by radiologists. Diagnosis rates vary widely among radiologists, and descriptive evidence suggests that a large component of this variation is due to differences in diagnostic skill. Our estimated model suggests that radiologists view failing to diagnose a patient with pneumonia as more costly than incorrectly diagnosing one without, and that this leads less skilled radiologists to optimally choose lower diagnostic thresholds. Variation in skill can explain 39% of the variation in diagnostic decisions, and policies that improve skill perform better than uniform decision guidelines. Failing to account for skill variation can lead to highly misleading results in research designs that use agent assignments as instruments.

SUBMITTER: Chan DC

PROVIDER: S-EPMC8992547 | biostudies-literature | 2022 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Selection with Variation in Diagnostic Skill: Evidence from Radiologists.

Chan David C DC Gentzkow Matthew M Yu Chuan C

The quarterly journal of economics 20220121 2

Physicians, judges, teachers, and agents in many other settings differ systematically in the decisions they make when faced with similar cases. Standard approaches to interpreting and exploiting such differences assume they arise solely from variation in preferences. We develop an alternative framework that allows variation in preferences and diagnostic skill and show that both dimensions may be partially identified in standard settings under quasi-random assignment. We apply this framework to s ...[more]

PMID: 35422677

Similar Datasets

Project description:PurposeTo examine radiologists' screening performance in relation to the number of diagnostic work-ups performed after abnormal findings are discovered at screening mammography by the same radiologist or by different radiologists.Materials and methodsIn an institutional review board-approved HIPAA-compliant study, the authors linked 651 671 screening mammograms interpreted from 2002 to 2006 by 96 radiologists in the Breast Cancer Surveillance Consortium to cancer registries (standard of reference) to evaluate the performance of screening mammography (sensitivity, false-positive rate [ FPR false-positive rate ], and cancer detection rate [ CDR cancer detection rate ]). Logistic regression was used to assess the association between the volume of recalled screening mammograms ("own" mammograms, where the radiologist who interpreted the diagnostic image was the same radiologist who had interpreted the screening image, and "any" mammograms, where the radiologist who interpreted the diagnostic image may or may not have been the radiologist who interpreted the screening image) and screening performance and whether the association between total annual volume and performance differed according to the volume of diagnostic work-up.ResultsAnnually, 38% of radiologists performed the diagnostic work-up for 25 or fewer of their own recalled screening mammograms, 24% performed the work-up for 0-50, and 39% performed the work-up for more than 50. For the work-up of recalled screening mammograms from any radiologist, 24% of radiologists performed the work-up for 0-50 mammograms, 32% performed the work-up for 51-125, and 44% performed the work-up for more than 125. With increasing numbers of radiologist work-ups for their own recalled mammograms, the sensitivity (P = .039), FPR false-positive rate (P = .004), and CDR cancer detection rate (P < .001) of screening mammography increased, yielding a stepped increase in women recalled per cancer detected from 17.4 for 25 or fewer mammograms to 24.6 for more than 50 mammograms. Increases in work-ups for any radiologist yielded significant increases in FPR false-positive rate (P = .011) and CDR cancer detection rate (P = .001) and a nonsignificant increase in sensitivity (P = .15). Radiologists with a lower annual volume of any work-ups had consistently lower FPR false-positive rate , sensitivity, and CDR cancer detection rate at all annual interpretive volumes.ConclusionThese findings support the hypothesis that radiologists may improve their screening performance by performing the diagnostic work-up for their own recalled screening mammograms and directly receiving feedback afforded by means of the outcomes associated with their initial decision to recall. Arranging for radiologists to work up a minimum number of their own recalled cases could improve screening performance but would need systems to facilitate this workflow.

Project description:Background and aimsTree diameter, tree height and canopy closure have been described by previous meta-analyses as being important characteristics in roost selection by cavity-roosting bats. However, size and direction of effects for these characteristics varied greatly among studies, also referred to as heterogeneity. Potential sources of heterogeneity have not been investigated in previous meta-analyses, which are explored by correlating additional covariates (moderator variables). We tested whether effect sizes from 34 studies were consistent enough to reject the null hypothesis that trees selected by bats did not significantly differ in their characteristics from randomly selected trees. We also examined whether heterogeneity in tree diameter effect sizes was correlated to moderator variables such as sex, bat species, habitat type, elevation and mean summer temperature.MethodsWe used Hedges' g standardized mean difference as the effect size for the most common characteristics that were encountered in the literature. We estimated heterogeneity indices, potential publication bias, and spatial autocorrelation of our meta-data. We relied upon meta-regression and multi-model inference approaches to evaluate the effects of moderator variables on heterogeneity in tree diameter effect sizes.ResultsTree diameter, tree height, snag density, elevation, and canopy closure were significant characteristics of roost selection by cavity-roosting bats. Size and direction of effects varied greatly among studies with respect to distance to water, tree density, slope, and bark remaining on trunks. Inclusion of mean summer temperature and sex in meta-regressions further explained heterogeneity in tree diameter effect sizes.ConclusionsRegional differences in roost selection for tree diameter were related to mean summer temperature. Large diameter trees play a central role in roost selection by bats, especially in colder regions, where they are likely to provide a warm and stable microclimate for reproductive females. Records of summer temperature fluctuations inside and outside tree cavities that are used by bats should be included in future research.

Dataset Information

Selection with Variation in Diagnostic Skill: Evidence from Radiologists.

Publications

Selection with Variation in Diagnostic Skill: Evidence from Radiologists.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets