Dataset Information

On the robustness of latent class models for diagnostic testing with no gold standard.

ABSTRACT: It is difficult to estimate sensitivity and specificity of diagnostic tests when there is no gold standard. Latent class models have been proposed as a potential solution as they provide estimates without the need for a gold standard. Using a motivating example of the evaluation of point of care tests for leptospirosis in Tanzania, we show how a realistic violation of assumptions underpinning the latent class model can lead directly to substantial bias in the estimates of the parameters of interest. In particular, we consider the robustness of estimates of sensitivity, specificity, and prevalence, to the presence of additional latent states when fitting a two-state latent class model. The violation is minor in the sense that it cannot be routinely detected with goodness-of-fit procedures, but is major with regard to the resulting bias.

SUBMITTER: Schofield MR

PROVIDER: S-EPMC8440412 | biostudies-literature | 2021 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

On the robustness of latent class models for diagnostic testing with no gold standard.

Schofield Matthew R MR Maze Michael J MJ Crump John A JA Rubach Matthew P MP Galloway Renee R Sharples Katrina J KJ

Statistics in medicine 20210514 22

It is difficult to estimate sensitivity and specificity of diagnostic tests when there is no gold standard. Latent class models have been proposed as a potential solution as they provide estimates without the need for a gold standard. Using a motivating example of the evaluation of point of care tests for leptospirosis in Tanzania, we show how a realistic violation of assumptions underpinning the latent class model can lead directly to substantial bias in the estimates of the parameters of inter ...[more]

PMID: 33990992

Similar Datasets

Project description:BackgroundIn Europe the principal definitive host for Echinococcus multilocularis, causing alveolar echinococcosis in humans, is the red fox (Vulpes vulpes). Obtaining reliable estimates of the prevalence of E. multilocularis and relevant risk factors for infection in foxes can be difficult if diagnostic tests with unknown test accuracies are used. Latent-class analysis can be used to obtain estimates of diagnostic test sensitivities and specificities in the absence of a perfect gold standard. Samples from 300 foxes in Switzerland were assessed by four different diagnostic tests including necropsy followed by sedimentation and counting technique (SCT), an egg-PCR, a monoclonal and a polyclonal copro-antigen ELISA. Information on sex, age and presence of other cestode species was assessed as potential covariates in the Bayesian latent class models. Different Bayesian latent-class models were run, considering dichotomized test results and, additionally, continuous readings resulting in empirical ROC curves.ResultsThe model without covariates estimated a true parasite prevalence of 59.5% (95% CI: 43.1-66.4%). SCT, assuming a specificity of 100%, performed best among the four tests with a sensitivity of 88.5% (95% CI: 82.7-93.4%). The egg-PCR showed a specificity of 93.4% (95% CI: 87.3-99.1%), although its sensitivity of 54.8% was found moderately low (95% CI: 48.5-61.0%). Relatively higher sensitivity (63.2%, 95% CI: 55.3-70.8%) and specificity (70.0%, 95% CI: 60.1-79.4%) were estimated for the monoclonal ELISA compared to the polyclonal ELISA with a sensitivity and specificity of 56.0% (95% CI: 48.0-63.9%) and 65.9% (95% CI: 55.8-75.6%), respectively. In the Bayesian models, adult foxes were found to be less likely infected than juveniles. Foxes with a concomitant cestode infection had double the odds of an E. multilocularis infection. ROC curves following a Bayesian approach enabled the empirical determination of the best cut-off point. While varying the cut-offs of both ELISAs, sensitivity and specificity of the egg-PCR and SCT remained constant in the Bayesian latent class models.ConclusionsAdoption of a Bayesian latent class approach helps to overcome the absence of a perfectly accurate diagnostic test and gives a more reliable indication of the test performance and the impact of covariates on the prevalence adjusted for diagnostic uncertainty.

Project description:BackgroundLatent class models can be used to estimate diagnostic accuracy without a gold standard test. Early studies often assumed independence between tests given the true disease state, however this can lead to biased estimates when there are inter-test dependencies. Residual correlation plots and chi-squared statistics have been commonly utilized to assess the validity of the conditional independence assumption and, when it does not hold, identify which test pairs are conditionally dependent. We aimed to assess the performance of these tools with a simulation study covering a wide range of scenarios.MethodsWe generated data sets from a model with four tests and a dependence between tests 1 and 2 within the diseased group. We varied sample size, prevalence, covariance, sensitivity and specificity, with 504 combinations of these in total, and 1000 data sets for each combination. We fitted the conditional independence model in a Bayesian framework, and reported absolute bias, coverage, and how often the residual correlation plots, G2 and χ2 statistics indicated lack-of-fit globally or for each test pair.ResultsAcross all settings, residual correlation plots, pairwise G2 and χ2 detected the correct correlated pair of tests only 12.1%, 10.3%, and 10.3% of the time, respectively, but incorrectly suggested dependence between tests 3 and 4 64.9%, 49.7%, and 49.5% of the time. We observed some variation in this across parameter settings, with these tools appearing to perform more as intended when tests 3 and 4 were both much more accurate than tests 1 and 2. Residual correlation plots, G2 and χ2 statistics identified a lack of overall fit in 74.3%, 64.5% and 67.5% of models, respectively. The conditional independence model tended to overestimate the sensitivities of the correlated tests (median bias across all scenarios 0.094, 2.5th and 97.5th percentiles -0.003, 0.397) and underestimate prevalence and the specificities of the uncorrelated tests.ConclusionsResidual correlation plots and chi-squared statistics cannot be relied upon to identify which tests are conditionally dependent, and also have relatively low power to detect lack of overall fit. This is important since failure to account for conditional dependence can lead to highly biased parameter estimates.

Dataset Information

On the robustness of latent class models for diagnostic testing with no gold standard.

Publications

On the robustness of latent class models for diagnostic testing with no gold standard.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets