Dataset Information

Statistical Use in Clinical Studies: Is There Evidence of a Methodological Shift?

ABSTRACT:

Background

Several studies indicate that the statistical education model and level in medical training fails to meet the demands of clinicians, especially when they want to understand published clinical research. We investigated how study designs and statistical methods in clinical studies have changed in the last twenty years, and we identified the current trends in study designs and statistical methods in clinical studies.

Methods

We reviewed 838 eligible clinical study articles that were published in 1990, 2000, and 2010 in four journals New England Journal of Medicine, Lancet, Journal of the American Medical Association and Nature Medicine. The study types, study designs, sample designs, data quality controls, statistical methods and statistical software were examined.

Results

Substantial changes occurred in the past twenty years. The majority of the studies focused on drug trials (61.6%, n = 516). In 1990, 2000, and 2010, there was an incremental increase in RCT studies (74.4%, 82.8%, and 84.0%, respectively, p = 0.013). Over time, there was increased attention on the details of selecting a sample and controlling bias, and there was a higher frequency of utilizing complex statistical methods. In 2010, the most common statistical methods were confidence interval for superiority and non-inferiority comparison (41.6%), survival analysis (28.5%), correction analysis for covariates (18.8%) and Logistic regression (15.3%).

Conclusions

These findings indicate that statistical measures in clinical studies are continuously developing and that the credibility of clinical study results is increasing. These findings provide information for future changes in statistical training in medical education.

SUBMITTER: Yi D

PROVIDER: S-EPMC4598157 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Statistical Use in Clinical Studies: Is There Evidence of a Methodological Shift?

Yi Dali D Ma Dihui D Li Gaoming G Zhou Liang L Xiao Qin Q Zhang Yanqi Y Liu Xiaoyu X Chen Hongru H Pettigrew Julia Christine JC Yi Dong D Liu Ling L Wu Yazhou Y

PloS one 20151008 10

<h4>Background</h4>Several studies indicate that the statistical education model and level in medical training fails to meet the demands of clinicians, especially when they want to understand published clinical research. We investigated how study designs and statistical methods in clinical studies have changed in the last twenty years, and we identified the current trends in study designs and statistical methods in clinical studies.<h4>Methods</h4>We reviewed 838 eligible clinical study articles ...[more]

PMID: 26448046

Similar Datasets

Project description:BackgroundThe evaluation of physical qualities in talent identification and development systems is vital and commonplace in supporting youth athletes towards elite sport. However, the complex and dynamic development of physical qualities in addition to temporal challenges associated with the research design, such as unstructured data collection and missing data, requires appropriate statistical methods to be applied in research to optimise the understanding and knowledge of long-term physical development.AimTo collate and evaluate the application of methodological and statistical methods used in studies investigating the development of physical qualities within youth athletes.MethodsElectronic databases were systematically searched form the earliest record to June 2021 and reference lists were hand searched in accordance with the PRISMA guidelines. Studies were included if they tested physical qualities over a minimum of 3 timepoints, were observational in nature and used youth sporting populations.ResultsForty articles met the inclusion criteria. The statistical analysis methods applied were qualitatively assessed against the theoretical underpinnings (i.e. multidimensional development, non-linear change and between and within athlete change) and temporal challenges (i.e. time variant and invariant variables, missing data, treatment of time and repeated measures) encountered with longitudinal physical testing research. Multilevel models were implemented most frequently (50%) and the most appropriately used statistical analysis method when qualitatively compared against the longitudinal challenges. Independent groups ANOVA, MANOVA and X2 were also used, yet failed to address any of the challenges posed within longitudinal physical testing research.ConclusionsThis methodological review identified the statistical methods currently employed within longitudinal physical testing research and addressed the theoretical and temporal challenges faced in longitudinal physical testing research with varying success. The findings can be used to support the selection of statistical methods when evaluating the development of youth athletes through the consideration of the challenges presented.

Project description:BackgroundWithin implementation science studies, contextual analysis is increasingly recognized as foundational to interventions' successful and sustainable implementation. However, inconsistencies between methodological approaches currently limit progress in studying context and guidance to standardize the use of those approaches is scant. Therefore, this study's objective was to systematically review and map current methodological approaches to contextual analysis in intervention implementation studies. The results would help us both to systematize the process of contextual analysis and identify gaps in the current evidence.MethodsWe conducted an evidence gap map (EGM) based on literature data via a stepwise approach. First, using an empirically developed search string, we randomly sampled 20% of all intervention implementation studies available from PubMed per year (2015-2020). Second, we assessed included studies that conducted a contextual analysis. Data extraction and evaluation followed the Basel Approach for CoNtextual ANAlysis (BANANA), using a color-coded rating scheme. Also based on BANANA and on the Context and Implementation of Complex Interventions (CICI) framework-an implementation framework that pays ample attention to context- we created visual maps of various approaches to contextual analysis.ResultsOf 15, 286 identified intervention implementation studies and study protocols, 3017 were screened for inclusion. Of those, 110 warranted close examination, revealing 22% that reported on contextual analysis. Only one study explicitly applied a framework for contextual analysis. Data were most commonly collected via surveys (n = 15) and individual interviews (n = 13). Ten studies reported mixed-methods analyses. Twenty-two assessed meso-level contextual and setting factors, with socio-cultural aspects most commonly studied. Eighteen described the use of contextual information for subsequent project phases (e.g., intervention development/adaption, selecting implementation strategies). Nine reported contextual factors' influences on implementation and/or effectiveness outcomes.ConclusionsThis study describes current approaches to contextual analysis in implementation science and provides a novel framework for evaluating and mapping it. By synthesizing our findings graphically in figures, we provide an initial evidence base framework that can incorporate new findings as necessary. We strongly recommend further development of methodological approaches both to conduct contextual analysis and to systematize the reporting of it. These actions will increase the quality and consistency of implementation science research.

Project description:BackgroundThere is substantial interest in the adaptation and application of so-called machine learning approaches to prognostic modelling of censored time-to-event data. These methods must be compared and evaluated against existing methods in a variety of scenarios to determine their predictive performance. A scoping review of how machine learning methods have been compared to traditional survival models is important to identify the comparisons that have been made and issues where they are lacking, biased towards one approach or misleading.MethodsWe conducted a scoping review of research articles published between 1 January 2000 and 2 December 2020 using PubMed. Eligible articles were those that used simulation studies to compare statistical and machine learning methods for risk prediction with a time-to-event outcome in a medical/healthcare setting. We focus on data-generating mechanisms (DGMs), the methods that have been compared, the estimands of the simulation studies, and the performance measures used to evaluate them.ResultsA total of ten articles were identified as eligible for the review. Six of the articles evaluated a method that was developed by the authors, four of which were machine learning methods, and the results almost always stated that this developed method's performance was equivalent to or better than the other methods compared. Comparisons were often biased towards the novel approach, with the majority only comparing against a basic Cox proportional hazards model, and in scenarios where it is clear it would not perform well. In many of the articles reviewed, key information was unclear, such as the number of simulation repetitions and how performance measures were calculated.ConclusionIt is vital that method comparisons are unbiased and comprehensive, and this should be the goal even if realising it is difficult. Fully assessing how newly developed methods perform and how they compare to a variety of traditional statistical methods for prognostic modelling is imperative as these methods are already being applied in clinical contexts. Evaluations of the performance and usefulness of recently developed methods for risk prediction should be continued and reporting standards improved as these methods become increasingly popular.

Project description:Multiple statistical approaches have been proposed to validate reference genes in qPCR assays. However, conflicting results from these statistical methods pose a major hurdle in the choice of the best reference genes. Recent studies have proposed the use of at least three different methods but there is no consensus on how to interpret conflicting results. Researchers resort to averaging the stability ranks assessed by different approaches or attributing a weighted rank to candidate genes. However, we report here that the suitability of these validation methods can be influenced by the experimental setting. Therefore, averaging the ranks can lead to suboptimal assessment of stable reference genes if the method used is not suitable for analysis. As the respective approaches of these statistical methods are different, a clear understanding of the fundamental assumptions and the parameters that influence the calculation of reference gene stability is necessary. In this study, the stability of 10 candidate reference genes (Actb, Gapdh, Tbp, Sdha, Pgk1, Ppia, Rpl13a, Hsp60, Mrpl10, Rps26) was assessed using four common statistical approaches (GeNorm, NormFinder, Coefficient of Variation or CV analysis and Pairwise ΔCt method) in a longitudinal experimental setting. We used the development of the cerebellum and the spinal cord of mice as a model to assess the suitability of these statistical methods for reference gene validation. GeNorm and the Pairwise ΔCt were found to be ill suited due to a fundamental assumption in their stability calculations. Highly correlated genes were given better stability ranks despite significant overall variation. NormFinder fares better but the presence of highly variable genes influences the ranking of all genes because of the algorithm's construct. CV analysis estimates overall variation, but it fails to consider variation across groups. We thus highlight the assumptions and potential pitfalls of each method using our longitudinal data. Based on our results, we have devised a workflow combining NormFinder, CV analysis along with visual representation of mRNA fold changes and one-way ANOVA for validating reference genes in longitudinal studies. This workflow proves to be more robust than any of these methods used individually.

Project description:BackgroundCalls have been made for greater application of the decision sciences to investigate and improve use of research evidence in mental health policy and practice. This article proposes a novel method, "decision sampling," to improve the study of decision-making and research evidence use in policy and programmatic innovation. An illustrative case study applies the decision sampling framework to investigate the decisions made by mid-level administrators when developing system-wide interventions to identify and treat the trauma of children entering foster care.MethodsDecision sampling grounds qualitative inquiry in decision analysis to elicit information about the decision-making process. Our case study engaged mid-level managers in public sector agencies (n = 32) from 12 states, anchoring responses on a recent index decision regarding universal trauma screening for children entering foster care. Qualitative semi-structured interviews inquired on questions aligned with key components of decision analysis, systematically collecting information on the index decisions, choices considered, information synthesized, expertise accessed, and ultimately the values expressed when selecting among available alternatives.ResultsFindings resulted in identification of a case-specific decision set, gaps in available evidence across the decision set, and an understanding of the values that guided decision-making. Specifically, respondents described 14 inter-related decision points summarized in five domains for adoption of universal trauma screening protocols, including (1) reach of the screening protocol, (2) content of the screening tool, (3) threshold for referral, (4) resources for screening startup and sustainment, and (5) system capacity to respond to identified needs. Respondents engaged a continuum of information that ranged from anecdote to research evidence, synthesizing multiple types of knowledge with their expertise. Policy, clinical, and delivery system experts were consulted to help address gaps in available information, prioritize specific information, and assess "fit to context." The role of values was revealed as participants evaluated potential trade-offs and selected among policy alternatives.ConclusionsThe decision sampling framework is a novel methodological approach to investigate the decision-making process and ultimately aims to inform the development of future dissemination and implementation strategies by identifying the evidence gaps and values expressed by the decision-makers, themselves.

Dataset Information

Statistical Use in Clinical Studies: Is There Evidence of a Methodological Shift?

Background

Methods

Results

Conclusions

Publications

Statistical Use in Clinical Studies: Is There Evidence of a Methodological Shift?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets