Dataset Information

The Moses-Littenberg meta-analytical method generates systematic differences in test accuracy compared to hierarchical meta-analytical models.

ABSTRACT: To compare meta-analyses of diagnostic test accuracy using the Moses-Littenberg summary receiver operating characteristic (SROC) approach with those of the hierarchical SROC (HSROC) model.Twenty-six data sets from existing test accuracy systematic reviews were reanalyzed with the Moses-Littenberg model, using equal weighting ("E-ML") and weighting by the inverse variance of the log DOR ("W-ML"), and with the HSROC model. The diagnostic odds ratios (DORs) were estimated and covariates added to both models to estimate relative DORs (RDORs) between subgroups. Models were compared by calculating the ratio of DORs, the ratio of RDORs, and P-values for detecting asymmetry and effects of covariates on DOR.Compared to the HSROC model, the Moses-Littenberg model DOR estimates were a median of 22% ("E-ML") and 47% ("W-ML") lower at Q*, and 7% and 42% lower at the central point in the data. Instances of the ML models giving estimates higher than the HSROC model also occurred. Investigations of heterogeneity also differed; the Moses-Littenberg models on average estimating smaller differences in RDOR.Moses-Littenberg meta-analyses can generate lower estimates of test accuracy, and smaller differences in accuracy, compared to mathematically superior hierarchical models. This has implications for the usefulness of meta-analyses using this approach. We recommend meta-analysis of diagnostic test accuracy studies to be conducted using available hierarchical model-based approaches.

SUBMITTER: Dinnes J

PROVIDER: S-EPMC5176007 | biostudies-other | 2016 Dec

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

The Moses-Littenberg meta-analytical method generates systematic differences in test accuracy compared to hierarchical meta-analytical models.

Dinnes Jacqueline J Mallett Susan S Hopewell Sally S Roderick Paul J PJ Deeks Jonathan J JJ

Journal of clinical epidemiology 20160730

<h4>Objective</h4>To compare meta-analyses of diagnostic test accuracy using the Moses-Littenberg summary receiver operating characteristic (SROC) approach with those of the hierarchical SROC (HSROC) model.<h4>Study design and setting</h4>Twenty-six data sets from existing test accuracy systematic reviews were reanalyzed with the Moses-Littenberg model, using equal weighting ("E-ML") and weighting by the inverse variance of the log DOR ("W-ML"), and with the HSROC model. The diagnostic odds rati ...[more]

PMID: 27485293

Similar Datasets

Project description:Primary studies examining the accuracy of a continuous test evaluate its sensitivity and specificity at one or more thresholds. Meta-analysts then usually perform a separate meta-analysis for each threshold. However, the number of studies available for each threshold is often very different, as primary studies are inconsistent in the thresholds reported. Furthermore, of concern is selective reporting bias, because primary studies may be less likely to report a threshold when it gives low sensitivity and/or specificity estimates. This may lead to biased meta-analysis results. We developed an exploratory method to examine the potential impact of missing thresholds on conclusions from a test accuracy meta-analysis.Our method identifies studies that contain missing thresholds bounded between a pair of higher and lower thresholds for which results are available. The bounded missing threshold results (two-by-two tables) are then imputed, by assuming a linear relationship between threshold value and each of logit-sensitivity and logit-specificity. The imputed results are then added to the meta-analysis, to ascertain if original conclusions are robust. The method is evaluated through simulation, and application made to 13 studies evaluating protein:creatinine ratio (PCR) for detecting proteinuria in pregnancy with 23 different thresholds, ranging from one to seven per study.The simulation shows the imputation method leads to meta-analysis estimates with smaller mean-square error. In the PCR application, it provides 50 additional results for meta-analysis and their inclusion produces lower test accuracy results than originally identified. For example, at a PCR threshold of 0.16, the summary specificity is 0.80 when using the original data, but 0.66 when also including the imputed data. At a PCR threshold of 0.25, the summary sensitivity is reduced from 0.95 to 0.85 when additionally including the imputed data.The imputation method is a practical tool for researchers (often non-statisticians) to explore the potential impact of missing threshold results on their meta-analysis conclusions. Software is available to implement the method. In the PCR example, it revealed threshold results are vulnerable to the missing data, and so stimulates the need for advanced statistical models or, preferably, individual patient data from primary studies.

Project description:Immunohistochemistry (IHC) and polymerase chain reaction (PCR) and fragment separation by capillary electrophoresis represent the current clinical laboratory standard for the evaluation of microsatellite instability (MSI) status. The importance of reporting MSI status in colorectal cancer is based on its potential for guiding treatment and as a prognostic indicator. It is also used to identify patients for Lynch syndrome testing. Our aim was to evaluate pre-analytical factors, such as age of formalin-fixed and paraffin-embedded (FFPE) block, neoplastic cell percentage, mucinous component, and DNA integrity, that may influence the accuracy of MSI testing and assess the concordance between three different MSI evaluation approaches. We selected the mucinous colorectal cancer (CRC) histotype for this study as it may possibly represent an intrinsic diagnostic issue due to its low tumor cellularity. Seventy-five cases of mucinous CRC and corresponding normal colon tissue samples were retrospectively selected. MMR proteins were evaluated by IHC. After DNA quality and quantity evaluation, the Idylla™ and TapeStation 4200 platforms were adopted for the evaluation of MSI status. Seventy-three (97.3%) cases were successfully analyzed by the three methodologies. Overall, the Idylla™ platform showed a concordance rate with IHC of 98.0% for microsatellite stable (MSS)/proficient MMR (pMMR) cases and 81.8% for MSI/deficient MMR (dMMR) cases. The TapeStation 4200 system showed a concordance rate with IHC of 96.0% for MSS/pMMR cases and 45.4% for MSI/dMMR cases. The concordance rates of the TapeStation 4200 system with respect to the Idylla™ platform were 98.1% for MSS profile and 57.8% for MSI profile. Discordant cases were analyzed using the Titano MSI kit. Considering pre-analytical factors, no significant variation in concordance rate among IHC analyses and molecular systems was observed by considering the presence of an acellular mucus cut-off >50% of the tumor area, FFPE year preparation, and DNA concentration. Conversely, the Idylla™ platform showed a significant variation in concordance rate with the IHC approach by considering a neoplastic cell percentage >50% (p-value = 0.002), and the TapeStation 4200 system showed a significant variation in concordance rate with the IHC approach by considering a DNA integrity number (DIN) ?4 as cut-off (p-value = 0.009). Our data pinpoint a central role of the pre-analytical phase in the diagnostic outcome of MSI testing in CRC.

Dataset Information

The Moses-Littenberg meta-analytical method generates systematic differences in test accuracy compared to hierarchical meta-analytical models.

Publications

The Moses-Littenberg meta-analytical method generates systematic differences in test accuracy compared to hierarchical meta-analytical models.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets