Dataset Information

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

ABSTRACT: Identifying recent HIV infection cases has important public health and clinical implications. It is essential for estimating incidence rates to monitor epidemic trends and evaluate the effectiveness of interventions. Detecting recent cases is also important for HIV prevention given the crucial role that recently infected individuals play in disease transmission, and because early treatment onset can improve the clinical outlook of patients while reducing transmission risk. Critical to this enterprise is the development and proper assessment of accurate classification assays that, based on cross-sectional samples of viral sequences, help determine infection recency status. In this work we assess some of the biases present in the evaluation of HIV recency classification algorithms that rely on measures of within-host viral diversity. Particularly, we examine how the time since infection (TSI) distribution of the infected subjects from which viral samples are drawn affect performance metrics (e.g., area under the ROC curve, sensitivity, specificity, accuracy and precision), potentially leading to misguided conclusions about the efficacy of classification assays. By comparing the performance of a given HIV recency assay using six different TSI distributions (four simulated TSI distributions representing different epidemic scenarios, and two empirical TSI distributions), we show that conclusions about the overall efficacy of the assay depend critically on properties of the TSI distribution. Moreover, we demonstrate that an assay with high overall classification accuracy, mainly due to properly sorting members of the well-represented groups in the validation dataset, can still perform notoriously poorly when sorting members of the less represented groups. This is an inherent issue of classification and diagnostics procedures that is often underappreciated. Thus, this work underscores the importance of acknowledging and properly addressing evaluation biases when proposing new HIV recency assays.

SUBMITTER: Patterson-Lomba O

PROVIDER: S-EPMC4593552 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

Patterson-Lomba Oscar O Wu Julia W JW Pagano Marcello M

PloS one 20151005 10

Identifying recent HIV infection cases has important public health and clinical implications. It is essential for estimating incidence rates to monitor epidemic trends and evaluate the effectiveness of interventions. Detecting recent cases is also important for HIV prevention given the crucial role that recently infected individuals play in disease transmission, and because early treatment onset can improve the clinical outlook of patients while reducing transmission risk. Critical to this enter ...[more]

PMID: 26436915

Similar Datasets

Project description:BackgroundHIV assays designed to detect recent infection, also known as "recency assays," are often used to estimate HIV incidence in a specific country, region, or subpopulation, alone or as part of recent infection testing algorithms (RITAs). Recently, many countries and organizations have become interested in using recency assays within case surveillance systems and routine HIV testing services to measure other indicators beyond incidence, generally referred to as "non-incidence surveillance use cases."ObjectiveThis review aims to identify published evidence that can be used to validate methodological approaches to recency-based incidence estimation and non-incidence use cases. The evidence identified through this review will be used in the forthcoming technical guidance by the World Health Organization (WHO) and United Nations Programme on HIV/AIDS (UNAIDS) on the use of HIV recency assays for identification of epidemic trends, whether for HIV incidence estimation or non-incidence indicators of recency.MethodsTo identify the best methodological and field implementation practices for the use of recency assays to estimate HIV incidence and trends in recent infections for specific populations or geographic areas, we conducted a systematic review of the literature to (1) understand the use of recency testing for surveillance in programmatic and laboratory settings, (2) review methodologies for implementing recency testing for both incidence estimation and non-incidence use cases, and (3) assess the field performance characteristics of commercially available recency assays.ResultsAmong the 167 documents included in the final review, 91 (54.5%) focused on assay or algorithm performance or methodological descriptions, with high-quality evidence of accurate age- and sex-disaggregated HIV incidence estimation at national or regional levels in general population settings, but not at finer geographic levels for prevention prioritization. The remaining 76 (45.5%) described the field use of incidence assays including field-derived incidence (n=45), non-incidence (n=25), and both incidence and non-incidence use cases (n=6). The field use of incidence assays included integrating RITAs into routine surveillance and assisting with molecular genetic analyses, but evidence was generally weaker or only reported on what was done, without validation data or findings related to effectiveness of using non-incidence indicators calculated through the use of recency assays as a proxy for HIV incidence.ConclusionsHIV recency assays have been widely validated for estimating HIV incidence in age- and sex-specific populations at national and subnational regional levels; however, there is a lack of evidence validating the accuracy and effectiveness of using recency assays to identify epidemic trends in non-incidence surveillance use cases. More research is needed to validate the use of recency assays within HIV testing services, to ensure findings can be accurately interpreted to guide prioritization of public health programming.

Project description:We previously described development of a rapid test for recent infection (RTRI) that can diagnose HIV infection and detect HIV-1 recent infections in a single device. This technology was transferred to a commercial partner as Asante Rapid Recency Assay (ARRA). We evaluated performance of the ARRA kits in the laboratory using a well-characterized panel of specimens. The plasma specimen panel (N = 1500) included HIV-1 (N = 570), HIV-2 (N = 10), and HIV-negatives (N = 920) representing multiple subtypes and geographic locations. Reference diagnostic data were generated using the Bio-Rad HIV-1-2-O EIA/Western blot algorithm with further serotyping performed using the Multispot HIV-1/2 assay. The LAg-Avidity EIA was used to generate reference data on recent and long-term infection for HIV-1 positive specimens at a normalized optical density (ODn) cutoff of 2.0 corresponding to a mean duration of about 6 months. All specimens were tested with ARRA according to the manufacturer's recommendations. Test strips were also read for line intensities using a reader and results were correlated with visual interpretation. ARRA's positive verification line (PVL) correctly classified 575 of 580 HIV-positive and 910 of 920 negative specimens resulting in a sensitivity of 99.1% (95% CI: 98.0-99.6) and specificity of 98.9% (95% CI: 98.1-99.4), respectively. The reader-based classification was similar for PVL with sensitivity of 99.3% (576/580) and specificity of 98.8% (909/920). ARRA's long-term line (LTL) classified 109 of 565 HIV-1 specimens as recent and 456 as long-term compared to 98 as recent and 467 as long-term (LT) by LAg-Avidity EIA (cutoff ODn = 2.0), suggesting a mean duration of recent infection (MDRI) close to 6 months. Agreement of ARRA with LAg recent cases was 81.6% (80/98) and LT cases was 93.8% (438/467), with an overall agreement of 91.7% (kappa = 0.72). The reader (cutoff 2.9) classified 109/566 specimens as recent infections compared to 99 by the LAg-Avidity EIA for recency agreement of 81.8% (81/99), LT agreement of 9% (439/467) with overall agreement of 91.9% (kappa = 0.72). The agreement between visual interpretation and strip reader was 99.9% (95% CI: 99.6-99.9) for the PVL and 98.1% (95% CI: 96.6-98.9) for the LTL. ARRA performed well with HIV diagnostic sensitivity >99% and specificity >98%. Its ability to identify recent infections is comparable to the LA-Avidity EIA corresponding to an MDRI of about 6 months. This point-of-care assay has implications for real-time surveillance of new infections among newly diagnosed individuals for targeted prevention and interrupting ongoing transmission thus accelerating epidemic control.

Dataset Information

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

Publications

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets