Correcting Diagnostic Test Sensitivity and Specificity for Patient Misclassifications Resulting from Use of an Imperfect Reference Standard
Ontology highlight
ABSTRACT:
SUBMITTER:
PROVIDER: S-EPMC9818831 | biostudies-literature | 2022 Dec
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Ignoring the fact that the reference test used to establish the discriminative properties of a combination of diagnostic biomarkers is imperfect can lead to a biased estimate of the diagnostic accuracy of the combination. In this paper, we propose a Bayesian latent-class mixture model to select a combination of biomarkers that maximizes the area under the ROC curve (AUC), while taking into account the imperfect nature of the reference test. In particular, a method for specification of the prior for the mixture component parameters is developed that allows controlling the amount of prior information provided for the AUC. The properties of the model are evaluated by using a simulation study and an application to real data from Alzheimer's disease research. In the simulation study, 100 data sets are simulated for sample sizes ranging from 100 to 600 observations, with a varying correlation between biomarkers. The inclusion of an informative as well as a flat prior for the diagnostic accuracy of the reference test is investigated. In the real-data application, the proposed model was compared with the generally used logistic-regression model that ignores the imperfectness of the reference test. Conditional on the selected sample size and prior distributions, the simulation study results indicate satisfactory performance of the model-based estimates. In particular, the obtained average estimates for all parameters are close to the true values. For the real-data application, AUC estimates for the proposed model are substantially higher than those from the 'traditional' logistic-regression model.
| S-EPMC6312185 | biostudies-literature
Project description:BackgroundStaquet et al. and Brenner both developed correction methods to estimate the sensitivity and specificity of a binary-response index test when the reference standard is imperfect and its sensitivity and specificity are known. However, to our knowledge, no study has compared the statistical properties of these methods, despite their long application in diagnostic accuracy studies.AimTo compare the correction methods developed by Staquet et al. and Brenner.MethodsSimulations techniques were employed to compare the methods under assumptions that the new test and the reference standard are conditionally independent or dependent given the true disease status of an individual. Three clinical datasets were analysed to understand the impact of using each method to inform clinical decision-making.ResultsUnder the assumption of conditional independence, the Staquet et al. correction method outperforms the Brenner correction method irrespective of the prevalence of disease and whether the performance of the reference standard is better or worse than the index test. However, when the prevalence of the disease is high (> 0.9) or low (< 0.1), the Staquet et al. correction method can produce illogical results (i.e. results outside [0,1]). Under the assumption of conditional dependence; both methods failed to estimate the sensitivity and specificity of the index test especially when the covariance terms between the index test and the reference standard is not close to zero.ConclusionWhen the new test and the imperfect reference standard are conditionally independent, and the sensitivity and specificity of the imperfect reference standard are known, the Staquet et al. correction method outperforms the Brenner method. However, where the prevalence of the target condition is very high or low or the two tests are conditionally dependent, other statistical methods such as latent class approaches should be considered.
| S-EPMC8040223 | biostudies-literature
Project description:BackgroundHIV self-testing (HIVST) has the potential to increase coverage of HIV testing, but concerns exist about intended users' ability to correctly perform and interpret tests, especially in poor communities with low literacy rates. We assessed the clinical performance of the 2016 prototype OraQuick® HIV Self-Test in rural and urban communities in Zambia to assess the sensitivity and specificity of the test compared to the national HIV rapid diagnostic test (RDT) algorithm and a laboratory reference standard using 4th generation enzyme immunoassays and HIV RNA detection.MethodsParticipants were recruited from randomly selected rural and urban households and one urban health facility between May 2016 and June 2017. Participants received a brief demonstration of the self-test, and then self-tested without further assistance. The research team re-read the self-test, repeated the self-test, drew blood for the laboratory reference, and conducted RDTs following the national HIV testing algorithm (Determine™ HIV1/2 (Alere) confirmed using Unigold™ HIV1/2 (Trinity Biotech)). Selected participants (N = 85) were videotaped whilst conducting the testing to observe common errors.ResultsInitial piloting showed that written instructions alone were inadequate, and a demonstration of self-test use was required. Of 2,566 self-test users, 2,557 (99.6%) were able to interpret their result. Of participants who were videoed 75/84 (89.3%) completed all steps of the procedure correctly. Agreement between the user-read result and the researcher-read result was 99.1%. Compared to the RDT algorithm, user-conducted HIVST was 94.1% sensitive (95%CI: 90.2-96.7) and 99.7% specific (95%CI: 99.3-99.9). Compared to the laboratory reference, both user-conducted HIVST (sensitivity 87.5%, 95%CI: 82.70-91.3; specificity 99.7%, 95%CI: 99.4-99.9) and the national RDT algorithm (sensitivity 93.4%, 95%CI: 89.7-96.1%; specificity 100% (95%CI: 99.8-100%) had considerably lower sensitivity.ConclusionsSelf-testers in Zambia who used OraQuick® HIV Self-Test achieved reasonable clinical performance compared to the national RDT algorithm. However, sensitivity of the self-test was reduced compared to a laboratory reference standard, as was the national RDT algorithm. In-person demonstration, along with the written manufacturer instructions, was needed to obtain accurate results. Programmes introducing self-care diagnostics should pilot and optimise support materials to ensure they are appropriately adapted to context.
| S-EPMC9134574 | biostudies-literature
Project description:People with sequence-space synesthesia (SSS) report stable visuo-spatial forms corresponding to numbers, days, and months (amongst others). This type of synesthesia has intrigued scientists for over 130 years but the lack of an agreed upon tool for assessing it has held back research on this phenomenon. The present study builds on previous tests by measuring the consistency of spatial locations that is known to discriminate controls from synesthetes. We document, for the first time, the sensitivity and specificity of such a test and suggest a diagnostic cut-off point for discriminating between the groups based on the area bounded by different placement attempts with the same item.
| S-EPMC5101286 | biostudies-literature
Project description:Animal trypanosomiasis (AT) is a significant livestock disease, affecting millions of animals across Sub-Saharan Africa, Central and South America, and Asia, and is caused by the protozoan parasites Trypanosoma brucei, Trypanosoma vivax, and Trypanosoma congolense, with the largest economic impact in cattle. There is over-reliance on presumptive chemotherapy due to inadequate existing diagnostic tests, highlighting the need for improved AT diagnostics. A small RNA species, the 7SL sRNA, is excreted/secreted by trypanosomes in infected animals, and has been previously shown to reliably diagnose active infection. We sought to explore key properties of 7SL sRNA RT-qPCR assays; namely, assessing the potential for cross-reaction with the widespread and benign Trypanosoma theileri, directly comparing assay performance against currently available diagnostic methods, quantitatively assessing specificity and sensitivity, and assessing the rate of decay of 7SL sRNA post-treatment. Results showed that the 7SL sRNA RT-qPCR assays specific for T. brucei, T. vivax, and T. congolense performed better than microscopy and DNA PCR in detecting infection. The 7SL sRNA signal was undetectable or significantly reduced by 96-h post treatment; at 1 × curative dose there was no detectable signal in 5/5 cattle infected with T. congolense, and in 3/5 cattle infected with T. vivax, with the signal being reduced 14,630-fold in the remaining two T. vivax cattle. Additionally, the assays did not cross-react with T. theileri. Finally, by using a large panel of validated infected and uninfected samples, the species-specific assays are shown to be highly sensitive and specific by receiver operating characteristic (ROC) analysis, with 100% sensitivity (95% CI, 96.44-100%) and 100% specificity (95% CI, 96.53-100%), 96.73% (95% CI, 95.54-99.96%) and 99.19% specificity (95% CI, 92.58-99.60%), and 93.42% (95% CI, 85.51-97.16% %) and 82.43% specificity (95% CI, 72.23-89.44% %) for the T brucei, T. congolense and T. vivax assays, respectively, under the conditions used. These findings indicate that the 7SL sRNA has many attributes that would be required for a potential diagnostic marker of AT: no cross-reaction with T. theileri, high specificity and sensitivity, early infection detection, continued signal even in the absence of detectable parasitaemia in blood, and clear discrimination between infected and treated animals.
| S-EPMC9017285 | biostudies-literature
Project description:BackgroundThe study aim is to estimate sensitivity and specificity of anal cytology for histologic HSIL in analyses adjusted for the imperfect biopsy reference standard.Methods and principal findingsRetrospective cohort study of an anal dysplasia screening program for HIV infected adults. We estimated the prevalence of histologic HSIL by concurrent cytology category and the associated cytology ROC area. Cytology operating characteristics for HSIL were estimated and adjusted for the imperfect reference standard by 3 methodologies. The study cohort included 261 patients with 3 available measures: (1) referral cytology; (2) HRA cytology; and (3) HRA directed biopsy. The prevalence of biopsy HSIL varied according to the concurrent HRA cytology result: 64.5% for HSIL or ASC-H, 12.6% for LSIL, 10.9% for ASCUS, and 6.3% for no abnormality. The cytology ROC area was 0.78. The observed prevalence of HSIL was 37% (referral cytology), 24% (HRA cytology), and 24% (HRA biopsy). Unadjusted estimates of sensitivity and specificity of cytology were 0.66 and 0.90, respectively. Adjusted estimates varied from 0.47-0.89 (sensitivity) and 0.89-1.0 (specificity).ConclusionsAnalysis of a single dataset yields widely different estimates of anal cytology operating characteristics that depend on difficult to verify assumptions regarding the accuracy of the imperfect reference standard.
| S-EPMC2924391 | biostudies-literature
Project description:The Coronavirus disease 2019 (COVID-19) pandemic-caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)- has posed a global threat and presented with it a multitude of economic and public-health challenges. Establishing a reliable means of readily available, rapid diagnostic testing is of paramount importance in halting the spread of COVID-19, as governments continue to ease lockdown restrictions. The current standard for laboratory testing utilizes reverse transcription quantitative polymerase chain reaction (RT-qPCR); however, this method presents clear limitations in requiring a longer run-time as well as reduced on-site testing capability. Therefore, we investigated the feasibility of a reverse transcription looped-mediated isothermal amplification (RT-LAMP)-based model of rapid COVID-19 diagnostic testing which allows for less invasive sample collection, named SaliVISION. This novel, two-step, RT-LAMP assay utilizes a customized multiplex primer set specifically targeting SARS-CoV-2 and a visual report system that is ready to interpret within 40 min from the start of sample processing and does not require a BSL-2 level testing environment or special laboratory equipment. When compared to the SalivaDirect and Thermo Fisher Scientific TaqPath RT-qPCR testing platforms, the respective sensitivities of the SaliVISION assay are 94.29% and 98.28% while assay specificity was 100% when compared to either testing platform. Our data illustrate a robust, rapid diagnostic assay in our novel RT-LAMP test design, with potential for greater testing throughput than is currently available through laboratory testing and increased on-site testing capability.
| S-EPMC8986854 | biostudies-literature
Project description:BackgroundSome emergency medical systems (EMS) use a dispatch centre where nurses or paramedics assess emergency calls and dispatch ambulances. Paramedics may also provide the first tier of care "in the field", with the second tier being an Emergency Physician (EP). In these systems, the appropriateness of the decision to dispatch an EP to the first line at the same time as the ambulance has not often been measured. The main objective of this study was to compare dispatching an EP as part of the first line emergency service with the severity of the patient's condition. The secondary objective was to highlight the need for a recognized reference standard to compare performance analyses across EMS.MethodsThis prospective observational study included all emergency calls received in Geneva's dispatch centre between January 1st, 2016 and June 30th, 2019. Emergency medical dispatchers (EMD) assigned a level of risk to patients at the time of the initial call. Only the highest level of risk led to the dispatch of an EP. The severity of the patient's condition observed in the field was measured using the National Advisory Committee for Aeronautics (NACA) scale. Two reference standards were proposed by dichotomizing the NACA scale. The first compared NACA≥4 with other conditions and the second compared NACA≥5 with other conditions. The level of risk identified during the initial call was then compared to the dichotomized NACA scales.Results97'861 assessments were included. Overall prevalence of sending an EP as first line was 13.11, 95% CI [12.90-13.32], and second line was 2.94, 95% CI [2.84-3.05]. Including NACA≥4, prevalence was 21.41, 95% CI [21.15-21.67], sensitivity was 36.2, 95% CI [35.5-36.9] and specificity 93.2 95% CI [93-93.4]. The Area Under the Receiver-Operating Characteristics curve (AUROC) of 0.7507, 95% CI [0.74734-0.75397] was acceptable. Looking NACA≥5, prevalence was 3.09, 95% CI [2.98-3.20], sensitivity was 64.4, 95% CI [62.7-66.1] and specificity 88.5, 95% CI [88.3-88.7]. We found an excellent AUROC of 0.8229, 95% CI [0.81623-0.82950].ConclusionThe assessment by Geneva's EMD has good specificity but low sensitivity for sending EPs. The dichotomy between immediate life-threatening and other emergencies could be a valid reference standard for future studies to measure the EP's dispatching performance.
| S-EPMC7871575 | biostudies-literature
Project description:Background : Leptospirosis is a growing public health concern in many tropical and subtropical countries. However, its diagnosis is difficult because of non-specific symptoms and concurrent other endemic febrile diseases. In many regions, the laboratory diagnosis is not available due to a lack of preparedness and simple diagnostic assay or difficult access to reference laboratories. Yet, an early antibiotic treatment is decisive to the outcome. The need for Rapid Diagnostic Tests (RDTs) for bedside diagnosis of leptospirosis has been recognized. We developed a vertical flow immunochromatography strip RDT detecting anti-Leptospira human IgM and evaluated it in patients from New Caledonia, France, and French West Indies. Methodology/Principal Findings : Whole killed Leptospira fainei cells were used as antigen for the test line and purified human IgM as the control line. The mobile phase was made of gold particles conjugated with goat anti-human IgM. Standards for Reporting of Diagnostic Accuracy criteria were used to assess the performance of this RDT. The Microscopic Agglutination Test (MAT) was used as the gold standard with a cut-off titer of ≥400. The sensitivity was 89.8% and the specificity 93.7%. Positive and negative Likelihood Ratios of 14.18 and 0.108 respectively, and a Diagnostic Odds Ratio of 130.737 confirmed its usefulness. This RDT had satisfactory reproducibility, repeatability, thermal tolerance and shelf-life. The comparison with MAT evidenced the earliness of the RDT to detect seroconversion. When compared with other RDT, the Vertical Flow RDT developed displayed good diagnostic performances.Conclusions/significanceThis RDT might be used as a point of care diagnostic tool in limited resources countries. An evaluation in field conditions and in other epidemiological contexts should be considered to assess its validity over a wider range of serogroups or when facing different endemic pathogens. It might prove useful in endemic contexts or outbreak situations.
| S-EPMC3694835 | biostudies-literature
Project description:BackgroundImperfect diagnostic testing reduces the power to detect significant predictors in classical cross-sectional studies. Assuming that the misclassification in diagnosis is random this can be dealt with by increasing the sample size of a study. However, the effects of imperfect tests in longitudinal data analyses are not as straightforward to anticipate, especially if the outcome of the test influences behaviour. The aim of this paper is to investigate the impact of imperfect test sensitivity on the determination of predictor variables in a longitudinal study.Methodology/principal findingsTo deal with imperfect test sensitivity affecting the response variable, we transformed the observed response variable into a set of possible temporal patterns of true disease status, whose prior probability was a function of the test sensitivity. We fitted a Bayesian discrete time survival model using an MCMC algorithm that treats the true response patterns as unknown parameters in the model. We applied our approach to epidemiological data of bovine tuberculosis outbreaks in England and investigated the effect of reduced test sensitivity in the determination of risk factors for the disease. We found that reduced test sensitivity led to changes to the collection of risk factors associated with the probability of an outbreak that were chosen in the 'best' model and to an increase in the uncertainty surrounding the parameter estimates for a model with a fixed set of risk factors that were associated with the response variable.Conclusions/significanceWe propose a novel algorithm to fit discrete survival models for longitudinal data where values of the response variable are uncertain. When analysing longitudinal data, uncertainty surrounding the response variable will affect the significance of the predictors and should therefore be accounted for either at the design stage by increasing the sample size or at the post analysis stage by conducting appropriate sensitivity analyses.
| S-EPMC3418251 | biostudies-literature