Dataset Information

A novel metric that quantifies risk stratification for evaluating diagnostic tests: The example of evaluating cervical-cancer screening tests across populations.

ABSTRACT: Our work involves assessing whether new biomarkers might be useful for cervical-cancer screening across populations with different disease prevalences and biomarker distributions. When comparing across populations, we show that standard diagnostic accuracy statistics (predictive values, risk-differences, Youden's index and Area Under the Curve (AUC)) can easily be misinterpreted. We introduce an intuitively simple statistic for a 2?×?2 table, Mean Risk Stratification (MRS): the average change in risk (pre-test vs. post-test) revealed for tested individuals. High MRS implies better risk separation achieved by testing. MRS has 3 key advantages for comparing test performance across populations with different disease prevalences and biomarker distributions. First, MRS demonstrates that conventional predictive values and the risk-difference do not measure risk-stratification because they do not account for test-positivity rates. Second, Youden's index and AUC measure only multiplicative relative gains in risk-stratification: AUC?=?0.6 achieves only 20% of maximum risk-stratification (AUC?=?0.9 achieves 80%). Third, large relative gains in risk-stratification might not imply large absolute gains if disease is rare, demonstrating a "high-bar" to justify population-based screening for rare diseases such as cancer. We illustrate MRS by our experience comparing the performance of cervical-cancer screening tests in China vs. the USA. The test with the worst AUC?=?0.72 in China (visual inspection with acetic acid) provides twice the risk-stratification (i.e. MRS) of the test with best AUC?=?0.83 in the USA (human papillomavirus and Pap cotesting) because China has three times more cervical precancer/cancer. MRS could be routinely calculated to better understand the clinical/public-health implications of standard diagnostic accuracy statistics.

SUBMITTER: Katki HA

PROVIDER: S-EPMC5851601 | biostudies-literature | 2018 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A novel metric that quantifies risk stratification for evaluating diagnostic tests: The example of evaluating cervical-cancer screening tests across populations.

Katki Hormuzd A HA Schiffman Mark M

Preventive medicine 20180214

Our work involves assessing whether new biomarkers might be useful for cervical-cancer screening across populations with different disease prevalences and biomarker distributions. When comparing across populations, we show that standard diagnostic accuracy statistics (predictive values, risk-differences, Youden's index and Area Under the Curve (AUC)) can easily be misinterpreted. We introduce an intuitively simple statistic for a 2 × 2 table, Mean Risk Stratification (MRS): the average change in ...[more]

PMID: 29454079

Similar Datasets

Project description:BackgroundIncreasing evidence supporting the value of screening women for cervical cancer once in their lifetime, coupled with mounting interest in scaling up successful screening demonstration projects, present challenges to public health decision makers seeking to take full advantage of the single-visit opportunity to provide additional services. We present an analytic framework for packaging multiple interventions during a single point of contact, explicitly taking into account a budget and scarce human resources, constraints acknowledged as significant obstacles for provision of health services in poor countries.Methods and findingsWe developed a binary integer programming (IP) model capable of identifying an optimal package of health services to be provided during a single visit for a particular target population. Inputs to the IP model are derived using state-transition models, which compute lifetime costs and health benefits associated with each intervention. In a simplified example of a single lifetime cervical cancer screening visit, we identified packages of interventions among six diseases that maximized disability-adjusted life years (DALYs) averted subject to budget and human resource constraints in four resource-poor regions. Data were obtained from regional reports and surveys from the World Health Organization, international databases, the published literature, and expert opinion. With only a budget constraint, interventions for depression and iron deficiency anemia were packaged with cervical cancer screening, while the more costly breast cancer and cardiovascular disease interventions were not. Including personnel constraints resulted in shifting of interventions included in the package, not only across diseases but also between low- and high-intensity intervention options within diseases.ConclusionsThe results of our example suggest several key themes: Packaging other interventions during a one-time visit has the potential to increase health gains; the shortage of personnel represents a real-world constraint that can impact the optimal package of services; and the shortage of different types of personnel may influence the contents of the package of services. Our methods provide a general framework to enhance a decision maker's ability to simultaneously consider costs, benefits, and important nonmonetary constraints. We encourage analysts working on real-world problems to shift from considering costs and benefits of interventions for a single disease to exploring what synergies might be achievable by thinking across disease burdens.

Dataset Information

A novel metric that quantifies risk stratification for evaluating diagnostic tests: The example of evaluating cervical-cancer screening tests across populations.

Publications

A novel metric that quantifies risk stratification for evaluating diagnostic tests: The example of evaluating cervical-cancer screening tests across populations.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets