Dataset Information

Increasing value and reducing waste in research design, conduct, and analysis.

ABSTRACT: Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.

SUBMITTER: Ioannidis JP

PROVIDER: S-EPMC4697939 | biostudies-literature | 2014 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Increasing value and reducing waste in research design, conduct, and analysis.

Ioannidis John P A JP Greenland Sander S Hlatky Mark A MA Khoury Muin J MJ Macleod Malcolm R MR Moher David D Schulz Kenneth F KF Tibshirani Robert R

Lancet (London, England) 20140108 9912

Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficie ...[more]

PMID: 24411645

Similar Datasets

Project description:BackgroundCompelling evidence has demonstrated that a large proportion of investment in biomedical research is wasted; this waste is avoidable. Academic institutions have, thus far, shown limited response to recommendations for increasing value and reducing waste. We formulated an academic response by (i) achieving consensus across a wide range of stakeholder groups on a comprehensive framework for quality of patient-oriented clinical research and (ii) highlighting first successful examples of its operationalization to facilitate waste-reducing strategies at academic institutions.Methods and findingsBased on a systematic review of quality definitions, concepts, and criteria in the medical literature (systematic MEDLINE search up to February 15, 2015, with independent and in duplicate article selection) and on stakeholder websites from 13 countries (Australia, Austria, Canada, France, Germany, Italy, Japan, Norway, Spain, Sweden, Switzerland, United Kingdom, and United States), we systematically developed a comprehensive framework for the quality of clinical research. We identified websites through personal contacts with experts in clinical research or public health who also suggested, for each country, websites of the following 7 stakeholder groups: patient organizations; academic research infrastructures; governmental bodies; regulatory agencies; ethics committees; the pharmaceutical industry; and funding agencies. In addition, we searched websites of inter- or supranational bodies involved in clinical research until no further insights emerged. After consolidation of the identified definitions, concepts, and criteria of quality in a basic framework structure, we conducted 4 rounds of an adapted online Delphi process among the same 7 stakeholder groups from 16 countries. The Delphi process ultimately achieved consensus on structure and content. The framework addresses 5 study stages (concept, planning and feasibility, conduct, analysis and interpretation, and reporting and knowledge translation) and includes the following dimensions: (i) protection of patient safety and rights, (ii) relevance/patient centeredness and involvement, (iii) minimization of bias (internal validity), (iv) precision, (v) transparency/access to data, and (vi) generalizability (external validity) of study results. These dimensions interact with 2 promoters-infrastructure and sustainability through education-that include a set of factors that may enhance all listed quality dimensions. Each quality dimension contains specific questions and explanatory items that guide quality assessment at each research stage from conceptualization of the research question through reporting and knowledge translation of study results. In the last survey round, Delphi participants from 9 countries (Austria, Australia, Canada, Germany, Italy, the Netherlands, Switzerland, UK, and US) agreed on the structure, content, and wording of the research stages, quality dimensions, specific questions, and descriptive examples of the final framework. In Switzerland, INQUIRE has resulted in a roadmap that guides initiatives to increase value within the Swiss Clinical Trial Organization network and through affiliated researchers.ConclusionsWe present a framework based on a consensus of different stakeholder groups guiding the practical assessment of clinical research quality at all stages of a research project. Operationalization of this common structure will support the increase of value by guiding academic institutions and researchers in developing quality enhancement initiatives, from posing the right research question to the transparent publication of results.

Project description:International experts have recommended actions that funders can take to improve the value of research investments. They state that self-assessment and public sharing are the basis for accountability and improvement. We examined our policies and practice to determine the extent to which the Patient-Centered Outcomes Research Institute's (PCORI) policies and practices as a research funder align with international best practice recommendations. A self-audit of current policies and practice against 17 recommendations and 35 sub-recommendations representing five major stages of research production, based on adapted methods used for self-assessment by another funder, was performed. Fit of existing PCORI policies and practices with 35 sub-recommendations, qualitative assessment of adequacy (area of strength; area of partial strength; area of growth; not applicable) for 17 recommendations for five stages of research production was assessed. Of the 17 recommendations, 15 were applicable to PCORI's research mission and focus. PCORI has policies and practices in place for all elements of six recommendations ("area of strength") and policies that address each element but with some still in active development for three ("area of partial strength"). PCORI is partially addressing six of the 15 relevant recommendations ("area of growth"). Areas for growth include making study protocols publicly available, improving policies on data sharing, and enhancing collaboration with other funders to reduce redundant funding. A voluntary consortium of international funders is underway to encourage further progress, including additional self-assessment and public sharing for accountability. These findings indicate PCORI has undertaken efforts to align its funding practices with international recommendations to ensure the value of public dollars invested in research. Further efforts will likely require additional coordination and collaboration between funders and stakeholders.

Dataset Information

Increasing value and reducing waste in research design, conduct, and analysis.

Publications

Increasing value and reducing waste in research design, conduct, and analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets