Dataset Information

Assessing intra-lab precision and inter-lab repeatability of outgrowth assays of HIV-1 latent reservoir size.

ABSTRACT: Quantitative viral outgrowth assays (QVOA) use limiting dilutions of CD4+ T cells to measure the size of the latent HIV-1 reservoir, a major obstacle to curing HIV-1. Efforts to reduce the reservoir require assays that can reliably quantify its size in blood and tissues. Although QVOA is regarded as a "gold standard" for reservoir measurement, little is known about its accuracy and precision or about how cell storage conditions or laboratory-specific practices affect results. Owing to this lack of knowledge, confidence intervals around reservoir size estimates-as well as judgments of the ability of therapeutic interventions to alter the size of the replication-competent but transcriptionally inactive latent reservoir-rely on theoretical statistical assumptions about dilution assays. To address this gap, we have carried out a Bayesian statistical analysis of QVOA reliability on 75 split samples of peripheral blood mononuclear cells (PBMC) from 5 antiretroviral therapy (ART)-suppressed participants, measured using four different QVOAs at separate labs, estimating assay precision and the effect of frozen cell storage on estimated reservoir size. We found that typical assay results are expected to differ from the true value by a factor of 1.6 to 1.9 up or down. Systematic assay differences comprised a 24-fold range between the assays with highest and lowest scales, likely reflecting differences in viral outgrowth readout and input cell stimulation protocols. We also found that controlled-rate freezing and storage of samples did not cause substantial differences in QVOA compared to use of fresh cells (95% probability of < 2-fold change), supporting continued use of frozen storage to allow transport and batched analysis of samples. Finally, we simulated an early-phase clinical trial to demonstrate that batched analysis of pre- and post-therapy samples may increase power to detect a three-fold reservoir reduction by 15 to 24 percentage points.

SUBMITTER: Rosenbloom DIS

PROVIDER: S-EPMC6481870 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Foot pain drawings (manikins) are commonly used to describe foot pain location in self-report health surveys. Respondents shade the manikin where they experience pain. The manikin is then scored via a transparent overlay that divides the drawings into areas. In large population based studies they are often scored by multiple raters. A difference in how different raters score manikins (inter-rater repeatability), or in how an individual rater scores manikins over time (intra-rater repeatability) can therefore affect data quality. This study aimed to assess inter- and intra-rater repeatability of scoring of the foot manikin.A random sample was generated of 50 respondents to a large population based survey of adults aged 50 years and older who experienced foot pain and completed a foot manikin. Manikins were initially scored by any one of six administrative staff (Rating 1). These manikins were re-scored by a second rater (Rating 2). The second rater then re-scored the manikins one week later (Rating 3). The following scores were compared: Rating 1 versus Rating 2 (inter-rater repeatability), and Rating 2 versus Rating 3 (intra-rater repeatability). A novel set of clinically relevant foot pain regions made up of one or more individual areas on the foot manikin were developed, and assessed for inter- and intra-rater repeatability.Scoring agreement of 100% (all 50 manikins) was seen in 69% (40 out of 58) of individual areas for inter-rater scoring (range 94 to 100%), and 81% (47 out of 58) of areas for intra-rater scoring (range 96 to 100%). All areas had a kappa value of ?0.70 for inter- and intra-rater scoring. Scoring agreement of 100% was seen in 50% (10 out of 20) of pain regions for inter-rater scoring (range 96 to 100%), and 95% (19 out of 20) of regions for intra-rater scoring (range 98 to 100%). All regions had a kappa value of >0.70 for inter- and intra-rater scoring.Individual and multiple raters can reliably score the foot pain manikin. In addition, our proposed regions may be used to reliably classify different patterns of foot pain using the foot manikin.

Project description:The aim of the present study is to evaluate the intra- and inter-observer agreement in assessing the renal function by means of 99mTc-MAG3 diuretic renography. One hundred and twenty adults were enrolled in the study. One experienced and one junior radiographer processed the renograms twice by assigning manual and semi-automated regions of interest. The differential renal function (DRF, %), time to maximum counts for the right and left kidney (TmaxR-TmaxL, min) and time to half-peak counts (T1/2, min) were calculated. The Bland-Altman analysis (bias±95% limits of agreement), Lin's concordance correlation coefficient and weighted Fleiss' kappa coefficient were used to assess agreement. Based on the Bland-Altman analysis, the intra-observer repeatability results for the experienced radiographer using the manual and the semi-automated techniques were 0.2 ± 2.6% and 0.3 ± 6.4% (DRF), respectively, -0.01 ± 0.24 and 0.00 ± 0.34 (TmaxR), respectively, and 0.00 ± 0.26 and 0.00 ± 0.33 (TmaxL), respectively. For the junior radiographer, the respective results were 0.5 ± 5.0% and 0.8 ± 9.4% (DRF), 0.00 ± 0.44 and 0.01 ± 0.28 (TmaxR), and 0.01 ± 0.28 and -0.02 ± 0.44 (TmaxL). The inter-observer repeatability for the manual method was 0.6 ± 5.0% (DRF), -0.10 ± 0.42 (TmaxR) and -0.05 ± 0.38 (TmaxL), and for the semi-automated method -0.2 ± 9.1% (DRF), 0.00 ± 0.31 (TmaxR) and -0.05 ± 0.40 (TmaxL). The weighted Fleiss' kappa coefficient for the T1/2 assessments ranged between 0.85-0.97 for both intra- and inter-observer repeatability with both methods. These findings suggest a very good repeatability in DRF assessment with the manual method-especially for the experienced observer-but a less good repeatability with the semi-automated approach. The calculation of Tmax was also operator-dependent. We conclude that reader experience is important in the calculation of renal parameters. We therefore encourage reader training in renal scintigraphy. Moreover, the manual tool seems to perform better than the semi-automated tool. Thus, we encourage cautious use of automated tools and adjunct validation by manual methods where possible.

Project description:Quantitative whole-body diffusion-weighted MRI (WB-DWI) is now possible using semi-automatic segmentation techniques. The method enables whole-body estimates of global Apparent Diffusion Coefficient (gADC) and total Diffusion Volume (tDV), both of which have demonstrated considerable utility for assessing treatment response in patients with bone metastases from primary prostate and breast cancers. Here we investigate the agreement (inter-observer repeatability) between two radiologists in their definition of Volumes Of Interest (VOIs) and subsequent assessment of tDV and gADC on an exploratory patient cohort of nine. Furthermore, each radiologist was asked to repeat his or her measurements on the same patient data sets one month later to identify the intra-observer repeatability of the technique. Using a Markov Chain Monte Carlo (MCMC) estimation method provided full posterior probabilities of repeatability measures along with maximum a-posteriori values and 95% confidence intervals. Our estimates of the inter-observer Intraclass Correlation Coefficient (ICCinter) for log-tDV and median gADC were 1.00 (0.97-1.00) and 0.99 (0.89-0.99) respectively, indicating excellent observer agreement for these metrics. Mean gADC values were found to have ICCinter = 0.97 (0.81-0.99) indicating a slight sensitivity to outliers in the derived distributions of gADC. Of the higher order gADC statistics, skewness was demonstrated to have good inter-user agreement with ICCinter = 0.99 (0.86-1.00), whereas gADC variance and kurtosis performed relatively poorly: 0.89 (0.39-0.97) and 0.96 (0.69-0.99) respectively. Estimates of intra-observer repeatability (ICCintra) demonstrated similar results: 0.99 (0.95-1.00) for log-tDV, 0.98 (0.89-0.99) and 0.97 (0.83-0.99) for median and mean gADC respectively, 0.64 (0.25-0.88) for gADC variance, 0.85 (0.57-0.95) for gADC skewness and 0.85 (0.57-0.95) for gADC kurtosis. Further investigation of two anomalous patient cases revealed that a very small proportion of voxels with outlying gADC values lead to instability in higher order gADC statistics. We therefore conclude that estimates of median/mean gADC and tumour volume demonstrate excellent inter- and intra-observer repeatability whilst higher order statistics of gADC should be used with caution when ascribing significance to clinical changes.

Dataset Information

Assessing intra-lab precision and inter-lab repeatability of outgrowth assays of HIV-1 latent reservoir size.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets