Dataset Information

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

ABSTRACT: Identifying recent HIV infection cases has important public health and clinical implications. It is essential for estimating incidence rates to monitor epidemic trends and evaluate the effectiveness of interventions. Detecting recent cases is also important for HIV prevention given the crucial role that recently infected individuals play in disease transmission, and because early treatment onset can improve the clinical outlook of patients while reducing transmission risk. Critical to this enterprise is the development and proper assessment of accurate classification assays that, based on cross-sectional samples of viral sequences, help determine infection recency status. In this work we assess some of the biases present in the evaluation of HIV recency classification algorithms that rely on measures of within-host viral diversity. Particularly, we examine how the time since infection (TSI) distribution of the infected subjects from which viral samples are drawn affect performance metrics (e.g., area under the ROC curve, sensitivity, specificity, accuracy and precision), potentially leading to misguided conclusions about the efficacy of classification assays. By comparing the performance of a given HIV recency assay using six different TSI distributions (four simulated TSI distributions representing different epidemic scenarios, and two empirical TSI distributions), we show that conclusions about the overall efficacy of the assay depend critically on properties of the TSI distribution. Moreover, we demonstrate that an assay with high overall classification accuracy, mainly due to properly sorting members of the well-represented groups in the validation dataset, can still perform notoriously poorly when sorting members of the less represented groups. This is an inherent issue of classification and diagnostics procedures that is often underappreciated. Thus, this work underscores the importance of acknowledging and properly addressing evaluation biases when proposing new HIV recency assays.

SUBMITTER: Patterson-Lomba O

PROVIDER: S-EPMC4593552 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

Patterson-Lomba Oscar O Wu Julia W JW Pagano Marcello M

PloS one 20151005 10

Identifying recent HIV infection cases has important public health and clinical implications. It is essential for estimating incidence rates to monitor epidemic trends and evaluate the effectiveness of interventions. Detecting recent cases is also important for HIV prevention given the crucial role that recently infected individuals play in disease transmission, and because early treatment onset can improve the clinical outlook of patients while reducing transmission risk. Critical to this enter ...[more]

PMID: 26436915

Similar Datasets

Project description:BackgroundHIV assays designed to detect recent infection, also known as "recency assays," are often used to estimate HIV incidence in a specific country, region, or subpopulation, alone or as part of recent infection testing algorithms (RITAs). Recently, many countries and organizations have become interested in using recency assays within case surveillance systems and routine HIV testing services to measure other indicators beyond incidence, generally referred to as "non-incidence surveillance use cases."ObjectiveThis review aims to identify published evidence that can be used to validate methodological approaches to recency-based incidence estimation and non-incidence use cases. The evidence identified through this review will be used in the forthcoming technical guidance by the World Health Organization (WHO) and United Nations Programme on HIV/AIDS (UNAIDS) on the use of HIV recency assays for identification of epidemic trends, whether for HIV incidence estimation or non-incidence indicators of recency.MethodsTo identify the best methodological and field implementation practices for the use of recency assays to estimate HIV incidence and trends in recent infections for specific populations or geographic areas, we conducted a systematic review of the literature to (1) understand the use of recency testing for surveillance in programmatic and laboratory settings, (2) review methodologies for implementing recency testing for both incidence estimation and non-incidence use cases, and (3) assess the field performance characteristics of commercially available recency assays.ResultsAmong the 167 documents included in the final review, 91 (54.5%) focused on assay or algorithm performance or methodological descriptions, with high-quality evidence of accurate age- and sex-disaggregated HIV incidence estimation at national or regional levels in general population settings, but not at finer geographic levels for prevention prioritization. The remaining 76 (45.5%) described the field use of incidence assays including field-derived incidence (n=45), non-incidence (n=25), and both incidence and non-incidence use cases (n=6). The field use of incidence assays included integrating RITAs into routine surveillance and assisting with molecular genetic analyses, but evidence was generally weaker or only reported on what was done, without validation data or findings related to effectiveness of using non-incidence indicators calculated through the use of recency assays as a proxy for HIV incidence.ConclusionsHIV recency assays have been widely validated for estimating HIV incidence in age- and sex-specific populations at national and subnational regional levels; however, there is a lack of evidence validating the accuracy and effectiveness of using recency assays to identify epidemic trends in non-incidence surveillance use cases. More research is needed to validate the use of recency assays within HIV testing services, to ensure findings can be accurately interpreted to guide prioritization of public health programming.

Dataset Information

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

Publications

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets