Dataset Information

Parametric methods outperformed non-parametric methods in comparisons of discrete numerical variables.

ABSTRACT: BACKGROUND: The number of events per individual is a widely reported variable in medical research papers. Such variables are the most common representation of the general variable type called discrete numerical. There is currently no consensus on how to compare and present such variables, and recommendations are lacking. The objective of this paper is to present recommendations for analysis and presentation of results for discrete numerical variables. METHODS: Two simulation studies were used to investigate the performance of hypothesis tests and confidence interval methods for variables with outcomes {0, 1, 2}, {0, 1, 2, 3}, {0, 1, 2, 3, 4}, and {0, 1, 2, 3, 4, 5}, using the difference between the means as an effect measure. RESULTS: The Welch U test (the T test with adjustment for unequal variances) and its associated confidence interval performed well for almost all situations considered. The Brunner-Munzel test also performed well, except for small sample sizes (10 in each group). The ordinary T test, the Wilcoxon-Mann-Whitney test, the percentile bootstrap interval, and the bootstrap-t interval did not perform satisfactorily. CONCLUSIONS: The difference between the means is an appropriate effect measure for comparing two independent discrete numerical variables that has both lower and upper bounds. To analyze this problem, we encourage more frequent use of parametric hypothesis tests and confidence intervals.

SUBMITTER: Fagerland MW

PROVIDER: S-EPMC3097007 | biostudies-other | 2011

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Parametric methods outperformed non-parametric methods in comparisons of discrete numerical variables.

Fagerland Morten W MW Sandvik Leiv L Mowinckel Petter P

BMC medical research methodology 20110413

<h4>Background</h4>The number of events per individual is a widely reported variable in medical research papers. Such variables are the most common representation of the general variable type called discrete numerical. There is currently no consensus on how to compare and present such variables, and recommendations are lacking. The objective of this paper is to present recommendations for analysis and presentation of results for discrete numerical variables.<h4>Methods</h4>Two simulation studies ...[more]

PMID: 21489231

Similar Datasets

Project description:BackgroundIn randomized clinical trials, treatment effects may vary, and this possibility is referred to as heterogeneity of treatment effect (HTE). One way to quantify HTE is to partition participants into subgroups based on individual's risk of experiencing an outcome, then measuring treatment effect by subgroup. Given the limited availability of externally validated outcome risk prediction models, internal models (created using the same dataset in which heterogeneity of treatment analyses also will be performed) are commonly developed for subgroup identification. We aim to compare different methods for generating internally developed outcome risk prediction models for subject partitioning in HTE analysis.MethodsThree approaches were selected for generating subgroups for the 2,441 participants from the United States enrolled in the ASPirin in Reducing Events in the Elderly (ASPREE) randomized controlled trial. An extant proportional hazards-based outcomes predictive risk model developed on the overall ASPREE cohort of 19,114 participants was identified and was used to partition United States' participants by risk of experiencing a composite outcome of death, dementia, or persistent physical disability. Next, two supervised non-parametric machine learning outcome classifiers, decision trees and random forests, were used to develop multivariable risk prediction models and partition participants into subgroups with varied risks of experiencing the composite outcome. Then, we assessed how the partitioning from the proportional hazard model compared to those generated by the machine learning models in an HTE analysis of the 5-year absolute risk reduction (ARR) and hazard ratio for aspirin vs. placebo in each subgroup. Cochran's Q test was used to detect if ARR varied significantly by subgroup.ResultsThe proportional hazard model was used to generate 5 subgroups using the quintiles of the estimated risk scores; the decision tree model was used to generate 6 subgroups (6 automatically determined tree leaves); and the random forest model was used to generate 5 subgroups using the quintiles of the prediction probability as risk scores. Using the semi-parametric proportional hazards model, the ARR at 5 years was 15.1% (95% CI 4.0-26.3%) for participants with the highest 20% of predicted risk. Using the random forest model, the ARR at 5 years was 13.7% (95% CI 3.1-24.4%) for participants with the highest 20% of predicted risk. The highest outcome risk group in the decision tree model also exhibited a risk reduction, but the confidence interval was wider (5-year ARR = 17.0%, 95% CI= -5.4-39.4%). Cochran's Q test indicated ARR varied significantly only by subgroups created using the proportional hazards model. The hazard ratio for aspirin vs. placebo therapy did not significantly vary by subgroup in any of the models. The highest risk groups for the proportional hazards model and random forest model contained 230 participants each, while the highest risk group in the decision tree model contained 41 participants.ConclusionsThe choice of technique for internally developed models for outcome risk subgroups influences HTE analyses. The rationale for the use of a particular subgroup determination model in HTE analyses needs to be explicitly defined based on desired levels of explainability (with features importance), uncertainty of prediction, chances of overfitting, and assumptions regarding the underlying data structure. Replication of these analyses using data from other mid-size clinical trials may help to establish guidance for selecting an outcomes risk prediction modelling technique for HTE analyses.

Project description:The potential of a metabolomics method to detect statistically significant perturbations in the metabolome of an organism is enhanced by excellent analytical precision, unequivocal identification, and broad metabolomic coverage. While the former two metrics are usually associated with targeted metabolomics and the latter with non-targeted metabolomics, a systematic comparison of the performance of both approaches has not yet been carried out. The present work reports on the development and performance evaluation of separate targeted and non-targeted metabolomics methods. The targeted approach facilitated determination of 181 metabolites (quantitative analysis of 18 amino acids, 11 biogenic amines, 5 neurotransmitters, 5 nucleobases and semi-quantitative analysis of 50 carnitines, 83 phosphatidylcholines, and 9 sphingomyelins) using ultra-performance liquid chromatography-tandem mass spectrometry (UPLC-MS/MS) and flow injection-tandem mass spectrometry (FI-MS/MS). Method accuracy and/or precision were assessed using replicate samples of NIST SRM1950 as well as fish liver and brain tissue from Gilthead Bream (Sparus aurata). The non-target approach involved UPLC-high resolution (Orbitrap) mass spectrometry (UPLC-HRMS). Testing of ionization mode and stationary phase revealed that a combination of positive electrospray ionization and HILIC chromatography produced the largest number of chromatographic features during non-target analysis. Furthermore, an evaluation of 4 different sequence drift correction algorithms, and combinations thereof, revealed that batchCorr produced the best precision in almost every test. However, even following correction of non-target data for signal drift, the precision of targeted data was better, confirming our existing assumptions about the strengths of targeted metabolomics. Finally, the accuracy of the online MS2-library mzCloud was evaluated using reference standards for 38 different metabolites. This is among the few studies that have systematically evaluated the performance of targeted and non-targeted metabolomics and provides new insight into the advantages and disadvantages of each approach.

Project description:BackgroundEnsuring fair comparisons of cancer survival statistics across population groups requires careful consideration of differential competing mortality due to other causes, and adjusting for imbalances over groups in other prognostic covariates (e.g. age). This has typically been achieved using comparisons of age-standardised net survival, with age standardisation addressing covariate imbalance, and the net estimates removing differences in competing mortality from other causes. However, these estimates lack ease of interpretability. In this paper, we motivate an alternative non-parametric approach that uses a common rate of other cause mortality across groups to give reference-adjusted estimates of the all-cause and cause-specific crude probability of death in contrast to solely reporting net survival estimates.MethodsWe develop the methodology for a non-parametric equivalent of standardised and reference adjusted crude probabilities of death, building on the estimation of non-parametric crude probabilities of death. We illustrate the approach using regional comparisons of survival following a diagnosis of rectal cancer for men in England. We standardise to the covariate distribution and other cause mortality of England as a whole to offer comparability, but with close approximation to the observed all-cause region-specific mortality.ResultsThe approach gives comparable estimates to observed crude probabilities of death, but allows direct comparison across population groups with different covariate profiles and competing mortality patterns. In our illustrative example, we show that regional variations in survival following a diagnosis of rectal cancer persist even after accounting for the variation in deprivation, age at diagnosis and other cause mortality.ConclusionsThe methodological approach of using standardised and reference adjusted metrics offers an appealing approach for future cancer survival comparison studies and routinely published cancer statistics. Our non-parametric estimation approach through the use of weighting offers the ability to estimate comparable survival estimates without the need for statistical modelling.

Dataset Information

Parametric methods outperformed non-parametric methods in comparisons of discrete numerical variables.

Publications

Parametric methods outperformed non-parametric methods in comparisons of discrete numerical variables.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets