Dataset Information

Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test.

ABSTRACT: Multiple comparisons tests (MCTs) include the statistical tests used to compare groups (treatments) often following a significant effect reported in one of many types of linear models. Due to a variety of data and statistical considerations, several dozen MCTs have been developed over the decades, with tests ranging from very similar to each other to very different from each other. Many scientific disciplines use MCTs, including >40,000 reports of their use in ecological journals in the last 60 years. Despite the ubiquity and utility of MCTs, several issues remain in terms of their correct use and reporting. In this study, we evaluated 17 different MCTs. We first reviewed the published literature for recommendations on their correct use. Second, we created a simulation that evaluated the performance of nine common MCTs. The tests examined in the simulation were those that often overlapped in usage, meaning the selection of the test based on fit to the data is not unique and that the simulations could inform the selection of one or more tests when a researcher has choices. Based on the literature review and recommendations: planned comparisons are overwhelmingly recommended over unplanned comparisons, for planned non-parametric comparisons the Mann-Whitney-Wilcoxon U test is recommended, Scheffé's S test is recommended for any linear combination of (unplanned) means, Tukey's HSD and the Bonferroni or the Dunn-Sidak tests are recommended for pairwise comparisons of groups, and that many other tests exist for particular types of data. All code and data used to generate this paper are available at: https://github.com/stevemidway/MultipleComparisons.

SUBMITTER: Midway S

PROVIDER: S-EPMC7720730 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test.

Midway Stephen S Robertson Matthew M Flinn Shane S Kaller Michael M

PeerJ 20201204

Multiple comparisons tests (MCTs) include the statistical tests used to compare groups (treatments) often following a significant effect reported in one of many types of linear models. Due to a variety of data and statistical considerations, several dozen MCTs have been developed over the decades, with tests ranging from very similar to each other to very different from each other. Many scientific disciplines use MCTs, including >40,000 reports of their use in ecological journals in the last 60 ...[more]

PMID: 33335808

Similar Datasets

Project description:Dermatomyositis is a heterogeneous idiopathic inflammatory myopathy associated with various cutaneous manifestations and variable presence of myositis, interstitial lung disease, and other visceral organ involvement. An accurate diagnosis of dermatomyositis requires correlating clinical examination findings with serological and histological findings. Familiarity with pathognomonic and common cutaneous manifestations of dermatomyositis, which are highlighted here, can be especially helpful in making an accurate diagnosis. Additionally, evaluating patients for presence of myositis-specific autoantibodies can further support or refute a dermatomyositis diagnosis. When present, myositis-specific autoantibodies can also help guide workups for various dermatomyositis-associated manifestations, as each is associated with relatively distinct clinical characteristics. Evaluating patients for various systemic manifestations often relies on expert opinion recommendations; however, societal guideline statements concerning the evaluation of some manifestations have recently been described. Although malignancy-associated dermatomyositis is a well-accepted subtype, there is limited evidence to support extensive malignancy screening has a favorable benefit-risk ratio in most dermatomyositis patients. However, recent research has uncovered novel associations between dermatomyositis and malignancy, suggesting the possibility of identifying high-risk subsets of dermatomyositis patients in whom malignancy screening may have a high value. Treatment for dermatomyositis has remained largely unchanged over the past several decades. Although many dermatomyositis patients can be effectively treated with current options, either as monotherapy or with combination regimens, there is a need for more targeted and effective DM therapies, in general, and for MDA5(+) dermatomyositis-associated rapidly progressive interstitial lung disease. Fortunately, significant current and emerging research activities evaluating various novel medications for dermatomyositis provide hope for exciting future advances in patients with this intriguing immune-mediated disease.

Project description:Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Dataset Information

Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test.

Publications

Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets