Quantifying how diagnostic test accuracy depends on threshold in a meta-analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Jones HE
PROVIDER: S-EPMC6856843 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND: Several types of statistical methods are currently available for the meta-analysis of studies on diagnostic test accuracy. One of these methods is the Bivariate Model which involves a simultaneous analysis of the sensitivity and specificity from a set of studies. In this paper, we review the characteristics of the Bivariate Model and demonstrate how it can be extended with a discrete latent variable. The resulting clustering of studies yields additional insight into the accuracy of the test of interest. METHODS: A Latent Class Bivariate Model is proposed. This model captures the between-study variability in sensitivity and specificity by assuming that studies belong to one of a small number of latent classes. This yields both an easier to interpret and a more precise description of the heterogeneity between studies. Latent classes may not only differ with respect to the average sensitivity and specificity, but also with respect to the correlation between sensitivity and specificity. RESULTS: The Latent Class Bivariate Model identifies clusters of studies with their own estimates of sensitivity and specificity. Our simulation study demonstrated excellent parameter recovery and good performance of the model selection statistics typically used in latent class analysis. Application in a real data example on coronary artery disease showed that the inclusion of latent classes yields interesting additional information. CONCLUSIONS: Our proposed new meta-analysis method can lead to a better fit of the data set of interest, less biased estimates and more reliable confidence intervals for sensitivities and specificities. But even more important, it may serve as an exploratory tool for subsequent sub-group meta-analyses.
| S-EPMC4105799 | biostudies-other
Project description:ImportanceRed reflex testing is a simple and inexpensive method implemented in many countries as an important part of infant screening for ocular pathologies.ObjectivesTo review the literature on the diagnostic accuracy of the red reflex test in infant screening for ocular pathologies and to perform meta-analyses to provide summary estimates.Data sourcesThe following literature databases were searched for English-language, peer-reviewed literature, published until April 19, 2020: Cochrane Central, PubMed/MEDLINE, Embase, Web of Science Core Collection, BIOSIS Previews, Current Contents Connect, Data Citation Index, Derwent Innovations Index, KCI-Korean Journal Database, Russian Science Citation Index, SciELO Citation Index, CINAHL (Cumulative Index to Nursing and Allied Health Literature), and ClinicalTrials.gov.Study selectionEligibility criteria were defined according to population (studies of consecutively screened infants), exposure (red reflex or Brückner test as the index test), comparator (any ophthalmological examination), and study type (any study with diagnostic test accuracy data).Data extraction and synthesisThe Preferred Reporting Items for Systematic Reviews and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines were followed. Data were extracted independently by 2 authors. For summary estimates of diagnostic test accuracy, the hierarchical summary receiver operating characteristics curve was used. Prevalence of ocular pathologies was introduced for a prevalence meta-analysis, which was then used in calculations of diagnostic accuracy of the red reflex test when applied in infant screening.Main outcomes and measuresTrue-positive, false-positive, true-negative, and false-negative findings; sensitivity; specificity; and positive and negative predictive values.ResultsIn this meta-analysis, 8713 unique infants from 5 unique studies were eligible for qualitative and quantitative review. All studies used the red reflex test without pupillary dilation and were compared with a reference test performed with pupillary dilation. For any ocular pathology, an estimated sensitivity of 7.5% (95% CI, 7.4%-7.5%) and specificity of 97.5% (95% CI, 97.5%-97.5%) was found. Focusing on ocular pathologies that required a medical or surgical intervention, sensitivity improved to 17.5% (95% CI, 0.8%-84.8%) and specificity remained high at 97.6% (95% CI, 87.7%-99.6%).Conclusions and relevanceThese findings suggest that an abnormal red reflex finding most likely reflects an underlying ocular pathology. However, a normal red reflex finding during screening does not exclude ocular disease.
| S-EPMC7662495 | biostudies-literature
Project description:Mutation of p110 alpha-catalytic subunit of phosphatidylinositol 3-kinase (PIK3CA) has high predictive and prognostic values for breast cancer. Hence, there has been a marked interest in detecting and monitoring PIK3CA genotype with non-invasive technique, such as circulating free DNA (cfDNA). However, the diagnostic accuracy of PIK3CA genotyping by cfDNA is still a problem of controversy. Here, we conducted the first meta-analysis to evaluate overall diagnostic performance of cfDNA for PIK3CA mutation detection. Literature search was performed in Pubmed, Embase and Cochrane Central Register of Controlled Trials databases. Seven cohorts from five studies with 247 patients were included. The pooled sensitivity, specificity, positive and negative likelihood ratio, diagnostic odds ratio and area under summary receiver operating characteristic curve were calculated for accuracy evaluation. The pooled sensitivity and specificity were 0.86 (95% confidence interval [CI] 0.32-0.99) and 0.98 (95% CI 0.86-1.00), respectively; the pooled positive and negative likelihood ratio were 42.8 (95% CI 5.1-356.9) and 0.14 (95% CI 0.02-1.34), respectively; diagnostic odds ratio for evaluating the overall diagnostic performance was 300 (95% CI 8-11867); area under summary receiver operating characteristic curve reached 0.99 (95% CI 0.97-0.99). Subgroup analysis with metastatic breast cancer revealed remarkable improvement in diagnostic performance (sensitivity: 0.86-0.91; specificity: 0.98; diagnostic odds ratio: 300-428). This meta-analysis proved that detecting PIK3CA gene mutation by cfDNA has high diagnostic accuracy in breast cancer, especially for metastatic breast cancer. It may serve as a reliable non-invasive assay for detecting and monitoring PIK3CA mutation status in order to deliver personalized and precise treatment.
| S-EPMC4918940 | biostudies-literature
Project description:Meta-analysis of diagnostic test accuracy often involves mixture of case-control and cohort studies. The existing bivariate random-effects models, which jointly model bivariate accuracy indices (e.g., sensitivity and specificity), do not differentiate cohort studies from case-control studies and thus do not utilize the prevalence information contained in the cohort studies. The recently proposed trivariate generalized linear mixed-effects models are only applicable to cohort studies, and more importantly, they assume a common correlation structure across studies and trivariate normality on disease prevalence, test sensitivity, and specificity after transformation by some pre-specified link functions. In practice, very few studies provide justifications of these assumptions, and sometimes these assumptions are violated. In this paper, we evaluate the performance of the commonly used random-effects model under violations of these assumptions and propose a simple and robust method to fully utilize the information contained in case-control and cohort studies. The proposed method avoids making the aforementioned assumptions and can provide valid joint inferences for any functions of overall summary measures of diagnostic accuracy. Through simulation studies, we find that the proposed method is more robust to model misspecifications than the existing methods. We apply the proposed method to a meta-analysis of diagnostic test accuracy for the detection of recurrent ovarian carcinoma. Copyright © 2016 John Wiley & Sons, Ltd.
| S-EPMC6143393 | biostudies-literature
Project description:Background and Aims: Cystic fibrosis-related liver disease (CFLD) is one of the leading causes of morbidity and mortality in cystic fibrosis (CF). Several non-invasive diagnostic methods have been proposed as screening tools for CFLD. Our aim was to rank all available non-invasive modalities for diagnostic performance. Methods: A systematic search was performed in five medical databases to find studies which reported on any single or composite non-invasive diagnostic test (as an index test) compared to the Debray, the EuroCare or the Colombo criteria (as a reference standard). Ranking was carried out with a Bayesian diagnostic test accuracy network meta-analysis based on superiority indices, calculated for pooled sensitivity (Se) and specificity (Sp) with a 95% confidence interval (CI). The study was registered under CRD42020155846 in PROSPERO. Results: Fifteen studies with 15 index tests and a combination of them were included. The New criteria proposed by Koh et al. - which represent a composite diagnostic definition for CFLD including liver biochemistry, ultrasonography, transient elastography and fibrosis markers-had the best performance for detecting CFLD (Se:94%[CI:58-100], Sp:72%[CI:52-84]); while transient elastography (Se:65%[CI:56-74], Sp:88%[CI:84-91]) and a combination of it with a tissue inhibitor of metalloproteinase-4 measurement (Se:78%[CI:30-100], Sp:64%[CI:18-95%]) proved to be the second and third best options, respectively. In the imaging techniques subgroup, transient elastography (Se:66%[CI:57-72], Sp:88%[CI:85-91%]), acoustic radiation force impulse in the right lobe (Se:54%[CI:33-74], Sp:88%[CI:66-96]) and that in the left lobe (Se:55%[CI:23-81], Sp:82%[CI:50-95]) were ranked the highest. Comparing biochemical markers/fibrosis indices, the measurement of the Forns index (Se:72%[CI:25-99], Sp:63%[CI:16-94]), the aspartate aminotransferase-to-platelet ratio (Se:55%[CI:41-68], Sp:83%[CI:66-89]) and alkaline phosphatase (Se:63%[CI:18-93], Sp:64%[CI:19-95]) were ranked the highest. Conclusion: The New criteria show the best diagnostic performance. In clinical practice, transient elastography seems to be a simple, cheap and non-invasive tool, outperforming imaging, biochemical and fibrosis tests for detecting CFLD. Further studies are needed to validate our findings.
| S-EPMC8353091 | biostudies-literature
Project description:Melanoma has the highest mortality rate among skin cancers, and early-diagnosis is essential to maximize survival rate. The current procedure for melanoma diagnosis is based on dermoscopy, i.e., a qualitative visual inspection of lesions with intrinsic limited diagnostic reliability and reproducibility. Other non-invasive diagnostic techniques may represent valuable solutions to retrieve additional objective information of a lesion. This review aims to compare the diagnostic performance of non-invasive techniques, alternative to dermoscopy, for melanoma detection in clinical settings. A systematic review of the available literature was performed using PubMed, Scopus and Google scholar databases (2010-September 2020). All human, in-vivo, non-invasive studies using techniques, alternative to dermoscopy, for melanoma diagnosis were included with no restriction on the recruited population. The reference standard was histology but dermoscopy was accepted only in case of benign lesions. Attributes of the analyzed studies were compared, and the quality was evaluated using CASP Checklist. For studies in which the investigated technique was implemented as a diagnostic tool (DTA studies), the QUADAS-2 tool was applied. For DTA studies that implemented a melanoma vs. other skin lesions classification task, a meta-analysis was performed reporting the SROC curves. Sixty-two references were included in the review, of which thirty-eight were analyzed using QUADAS-2. Study designs were: clinical trials (13), retrospective studies (10), prospective studies (8), pilot studies (10), multitiered study (1); the remain studies were proof of concept or had undefined study type. Studies were divided in categories based on the physical principle employed by each diagnostic technique. Twenty-nine out of thirty-eight DTA studies were included in the meta-analysis. Heterogeneity of studies' types, testing strategy, and diagnostic task limited the systematic comparison of the techniques. Based on the SROC curves, spectroscopy achieved the best performance in terms of sensitivity (93%, 95% CI 92.8-93.2%) and specificity (85.2%, 95%CI 84.9-85.5%), even though there was high concern regarding robustness of metrics. Reflectance-confocal-microscopy, instead, demonstrated higher robustness and a good diagnostic performance (sensitivity 88.2%, 80.3-93.1%; specificity 65.2%, 55-74.2%). Best practice recommendations were proposed to reduce bias in future DTA studies. Particular attention should be dedicated to widen the use of alternative techniques to conventional dermoscopy.
| S-EPMC8103840 | biostudies-literature
Project description:We performed this systematic review and meta-analysis to assess the diagnostic accuracy of detecting glutamate dehydrogenase (GDH) for Clostridium difficile infection (CDI) based on the hierarchical model. Two investigators electrically searched four databases. Reference tests were stool cell cytotoxicity neutralization assay (CCNA) and stool toxigenic culture (TC). To assess the overall accuracy, we calculated the diagnostic odds ratio (DOR) using a DerSimonian-Laird random-model and area the under hierarchical summary receiver operating characteristics (AUC) using Holling's proportional hazard models. The summary estimate of the sensitivity and the specificity were obtained using the bivariate model. According to 42 reports consisting of 3055 reference positive comparisons, and 26188 reference negative comparisons, the DOR was 115 (95%CI: 77-172, I(2) = 12.0%) and the AUC was 0.970 (95%CI: 0.958-0.982). The summary estimate of sensitivity and specificity were 0.911 (95%CI: 0.871-0.940) and 0.912 (95%CI: 0.892-0.928). The positive and negative likelihood ratios were 10.4 (95%CI 8.4-12.7) and 0.098 (95%CI 0.066-0.142), respectively. Detecting GDH for the diagnosis of CDI had both high sensitivity and specificity. Considering its low cost and prevalence, it is appropriate for a screening test for CDI.
| S-EPMC4945925 | biostudies-literature
Project description:Muscle ultrasound is an emerging tool for diagnosing sarcopenia. This review aims to summarize the current knowledge on the diagnostic test accuracy of ultrasound for the diagnosis of sarcopenia. We collected data from Ovid Medline, Embase and the Cochrane Central Register of Controlled Trials. Diagnostic test accuracy studies using muscle ultrasound to detect sarcopenia were included. Bivariate random-effects models based on sensitivity and specificity pairs were used to calculate the pooled estimates of sensitivity, specificity and the area under the curves (AUCs) of summary receiver operating characteristic (SROC), if possible. We screened 7332 publications and included 17 studies with 2143 participants (mean age range: 52.6-82.8 years). All included studies had a high risk of bias. The study populations, reference standards and ultrasound measurement methods varied across the studies. Lower extremity muscles were commonly studied, whereas muscle thickness (MT) was the most widely measured parameter, followed by the cross-sectional area (CSA). The MTs of the gastrocnemius, rectus femoris, tibialis anterior, soleus, rectus abdominis and geniohyoid muscles showed a moderate diagnostic accuracy for sarcopenia (SROC-AUC 0.83, 8 studies; SROC-AUC 0.78, 5 studies; AUC 0.82, 1 study; AUC 0.76-0.78, 2 studies; AUC 0.76, 1 study; and AUC 0.79, 1 study, respectively), whereas the MTs of vastus intermedius, quadriceps femoris and transversus abdominis muscles showed a low diagnostic accuracy (AUC 0.67-0.71, 3 studies; SROC-AUC 0.64, 4 studies; and AUC 0.68, 1 study, respectively). The CSA of rectus femoris, biceps brachii muscles and gastrocnemius fascicle length also showed a moderate diagnostic accuracy (AUC 0.70-0.90, 3 studies; 0.81, 1 study; and 0.78-0.80, 1 study, respectively), whereas the echo intensity (EI) of rectus femoris, vastus intermedius, quadriceps femoris and biceps brachii muscles showed a low diagnostic accuracy (AUC 0.52-0.67, 2 studies; 0.48-0.50, 1 study; 0.43-0.49, 1 study; and 0.69, 1 study, respectively). The combination of CSA and EI of biceps brachii or rectus femoris muscles was better than either CSA or EI alone for diagnosing sarcopenia. Muscle ultrasound shows a low-to-moderate diagnostic test accuracy for sarcopenia diagnosis depending on different ultrasound parameters, measured muscles, reference standards and study populations. The combination of muscle quality indicators (e.g., EI) and muscle quantity indicators (e.g., MT) might provide better diagnostic test accuracy.
| S-EPMC9891970 | biostudies-literature
Project description:Background & aimsQuantitative fecal immunochemical tests (FITs) for hemoglobin are commonly used for colorectal cancer (CRC) screening. We aimed to quantify the change in CRC and advanced adenoma detection and number of positive test results at different positivity thresholds and by sex and age.MethodsWe searched MEDLINE and EMBASE, selecting articles of FIT for CRC detection in asymptomatic adults undergoing screening. We calculated sensitivity and specificity, as well as detected number of cancers, advanced adenomas, and positive test results at positivity thresholds ≤10 μg hemoglobin/g feces, 10 to ≤20 μg/g, 20 to ≤30 μg/g, and >30 μg/g. We also analyzed results from stratified by patient sex, age, and reference standard.ResultsOur meta-analysis comprised 46 studies with 2.4 million participants and 6478 detected cancers. Sensitivity for detection of CRC increased from 69% (95% confidence interval [CI], 63%-75%) at thresholds >10 μg/g and ≤20 μg/g to 80% (95% CI, 76%-83%) at thresholds ≤10 μg/g. At these threshold values, sensitivity for detection of advanced adenomas increased from 21% (95% CI, 18%-25%) to 31% (95% CI, 27%-35%), whereas specificity decreased from 94% (95% CI, 93%-96%) to 91% (95% CI, 89%-93%). In 3 studies stratified by sex, sensitivity of CRC detection was 77% in men (95% CI, 75%-79%) and 81% in women (95% CI, 60%-100%) (P = .68). In 3 studies stratified by age groups, sensitivity of CRC detection was 85% for ages 50-59 years (95% CI, 71%-99%) and 73% for ages 60-69 years (95% CI, 71%-75%) (P = .10). All studies with colonoscopy follow-up had similar sensitivity levels for detection of CRC to studies that analyzed 2-year registry follow-up data (74%; 95% CI, 68%-78% vs 75%; 95% CI, 73%-77%).ConclusionsIn a meta-analysis of studies that analyzed detection of CRC and advanced adenomas at different FIT positivity thresholds, we found the sensitivity and specificity of detection to vary with positive cutoff value. It might be possible to decrease positive threshold values for centers with sufficient follow-up colonoscopy resources. More research is needed to precisely establish FIT thresholds for each sex and age subgroup.ProtocolPROSPERO CRD42017068760.
| S-EPMC6878177 | biostudies-literature
Project description:Thiopurine S-methyltransferase (TPMT) deficiency increases the risk of serious adverse events in persons receiving thiopurines. The objective was to synthesize reported sensitivity and specificity of TPMT phenotyping and genotyping using a latent class hierarchical summary receiver operating characteristic meta-analysis. In 27 studies, pooled sensitivity and specificity of phenotyping for deficient individuals was 75.9% (95% credible interval (CrI), 58.3-87.0%) and 98.9% (96.3-100%), respectively. For genotype tests evaluating TPMT*2 and TPMT*3, sensitivity and specificity was 90.4% (79.1-99.4%) and 100.0% (99.9-100%), respectively. For individuals with deficient or intermediate activity, phenotype sensitivity and specificity was 91.3% (86.4-95.5%) and 92.6% (86.5-96.6%), respectively. For genotype tests evaluating TPMT*2 and TPMT*3, sensitivity and specificity was 88.9% (81.6-97.5%) and 99.2% (98.4-99.9%), respectively. Genotyping has higher sensitivity as long as TPMT*2 and TPMT*3 are tested. Both approaches display high specificity. Latent class meta-analysis is a useful method for synthesizing diagnostic test performance data for clinical practice guidelines.The Pharmacogenomics Journal advance online publication, 24 May 2016; doi:10.1038/tpj.2016.37.
| S-EPMC4957983 | biostudies-literature