Dataset Information

Published patterns of spin in biomedical literature: a protocol for a meta-research study

ABSTRACT:

Introduction

Spin is defined as reporting practices that distort the interpretation of results and create misleading conclusions by suggesting more favourable results. Such unjustifiable and misleading misrepresentation may negatively influence the development of further studies, clinical practice and healthcare policies. Spin manifests in various patterns in different sections of publications (titles, abstracts and main texts). The primary aim of this study is to identify reported spin patterns and assess the prevalence of spin in general, and the prevalence of spin patterns reported in biomedical literature based on previously published systematic reviews and literature reviews on spin.

Methods and analysis

PubMed, EMBASE and SCOPUS will be searched to identify systematic or literature reviews on spin in biomedicine. To improve the comprehensiveness of the search, the snowballing method will be used to broaden the search. The data on spin-related outcomes and characteristics of the included studies will be extracted. The methodological quality of the included studies will be assessed with selective items of the A MeaSurement Tool to Assess systematic Reviews-2 checklist. A new classification scheme for spin patterns will be developed if the classifications of spin patterns identified in the included studies vary. The prevalence of spin and spin patterns will be pooled based on meta-analyses if the classification schemes for spin are comparable across included studies. Otherwise, the prevalence will be described qualitatively. The seriousness of spin patterns will be assessed based on a Delphi consensus study.

Ethics and dissemination

This study has been approved by the Academic Centre for Dentistry Amsterdam Ethics Review Committee (2020250). The study will be submitted to a peer-reviewed scientific journal.

Registration

Open Science Framework: osf.io/hzv6e

SUBMITTER: Su N

PROVIDER: S-EPMC8039230 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Biomedical Research Centres (BRCs) are partnerships between healthcare organisations and universities in England. Their mission is to generate novel treatments, technologies, diagnostics and other interventions that increase the country's international competitiveness, to rapidly translate these innovations into benefits for patients, and to improve efficiency and reduce waste in healthcare. As NIHR Oxford BRC (Oxford BRC) enters its third 5-year funding period, we seek to (1) apply the evidence base on how best to support the various partnerships in this large, multi-stakeholder research system and (2) research how these partnerships play out in a new, ambitious programme of translational research.Organisational case study, informed by the principles of action research. A cross-cutting theme, 'Partnerships for Health, Wealth and Innovation' has been established with multiple sub-themes (drug development, device development, business support and commercialisation, research methodology and statistics, health economics, bioethics, patient and public involvement and engagement, knowledge translation, and education and training) to support individual BRC research themes and generate cross-theme learning. The 'Partnerships' theme will support the BRC's goals by facilitating six types of partnership (with patients and citizens, clinical services, industry, across the NIHR infrastructure, across academic disciplines, and with policymakers and payers) through a range of engagement platforms and activities. We will develop a longitudinal progress narrative centred around exemplar case studies, and apply theoretical models from innovation studies (Triple Helix), sociology of science (Mode 2 knowledge production) and business studies (Value Co-creation). Data sources will be the empirical research studies within individual BRC research themes (who will apply separately for NHS ethics approval), plus documentary analysis and interviews and ethnography with research stakeholders. This study has received ethics clearance through the University of Oxford Central University Research Ethics Committee.We anticipate that this work will add significant value to Oxford BRC. We predict accelerated knowledge translation; closer alignment of the innovation process with patient priorities and the principles of responsible, ethical research; reduction in research waste; new knowledge about the governance and activities of multi-stakeholder research partnerships and the contexts in which they operate; and capacity-building that reflects the future needs of a rapidly-evolving health research system.

Project description:OBJECTIVE:To measure the frequency of adequate methods, inadequate methods and poor reporting in published randomised controlled trials (RCTs) and test potential factors associated with adequacy of methods and reporting. DESIGN:Retrospective analysis of RCTs included in Cochrane reviews. Time series describes the proportion of RCTs using adequate methods, inadequate methods and poor reporting. A multinomial logit model tests potential factors associated with methods and reporting, including funding source, first author affiliation, clinical trial registration status, study novelty, team characteristics, technology and geography. DATA:Risk of bias assessments for random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data and selective reporting, for each RCT, were mapped to bibliometric and funding data. OUTCOMES:Risk of bias on six methodological dimensions and RCT-level overall assessment of adequate methods, inadequate methods or poor reporting. RESULTS:This study analysed 20 571 RCTs. 5.7% of RCTs used adequate methods (N=1173). 59.3% used inadequate methods (N=12 190) and 35.0% were poorly reported (N=7208). The proportion of poorly reported RCTs decreased from 42.5% in 1990 to 30.2% in 2015. The proportion of RCTs using adequate methods increased from 2.6% in 1990 to 10.3% in 2015. The proportion of RCTs using inadequate methods increased from 54.9% in 1990 to 59.5% in 2015. Industry funding, top pharmaceutical company affiliation, trial registration, larger authorship teams, international teams and drug trials were associated with a greater likelihood of using adequate methods. National Institutes of Health funding and university prestige were not. CONCLUSION:Even though reporting has improved since 1990, the proportion of RCTs using inadequate methods is high (59.3%) and increasing, potentially slowing progress and contributing to the reproducibility crisis. Stronger incentives for the use of adequate methods are needed.

Project description:BackgroundThere is an increasing need to evaluate the production and impact of medical research produced by institutions. Many indicators exist, yet we do not have enough information about their relevance. The objective of this systematic review was (1) to identify all the indicators that could be used to measure the output and outcome of medical research carried out in institutions and (2) enlist their methodology, use, positive and negative points.MethodologyWe have searched 3 databases (Pubmed, Scopus, Web of Science) using the following keywords: [Research outcome* OR research output* OR bibliometric* OR scientometric* OR scientific production] AND [indicator* OR index* OR evaluation OR metrics]. We included articles presenting, discussing or evaluating indicators measuring the scientific production of an institution. The search was conducted by two independent authors using a standardised data extraction form. For each indicator we extracted its definition, calculation, its rationale and its positive and negative points. In order to reduce bias, data extraction and analysis was performed by two independent authors.FindingsWe included 76 articles. A total of 57 indicators were identified. We have classified those indicators into 6 categories: 9 indicators of research activity, 24 indicators of scientific production and impact, 5 indicators of collaboration, 7 indicators of industrial production, 4 indicators of dissemination, 8 indicators of health service impact. The most widely discussed and described is the h-index with 31 articles discussing it.DiscussionThe majority of indicators found are bibliometric indicators of scientific production and impact. Several indicators have been developed to improve the h-index. This indicator has also inspired the creation of two indicators to measure industrial production and collaboration. Several articles propose indicators measuring research impact without detailing a methodology for calculating them. Many bibliometric indicators identified have been created but have not been used or further discussed.

Project description:IntroductionOne potential source of bias in randomised clinical trials of psychological interventions is researcher allegiance (RA). The operationalisation of RA differs strongly across studies, and there is not a generally accepted method of operationalising or measuring it. Furthermore, it remains unclear as to how RA affects the outcomes of trials and if it results in better outcomes for a preferred intervention. The aim of this project is to develop and validate a scale that accurately identifies RA, contribute to the understanding of the impact that RA has in a research setting and to make recommendations for addressing RA in practice.Methods and analysisA scale will first be developed and validated to measure RA in psychotherapy trials. The scale will be validated by surveying authors of psychotherapy trials to assess their opinions, beliefs and preferences of psychotherapy interventions. Furthermore, the scale will be validated for use outside the field of psychotherapy. The validated checklist will then be used to examine two potential mechanisms of how RA may affect outcomes of interventions: publication bias (by assessing grants) and risk of bias (RoB). Finally, recommendations will be developed, and a feasibility study will be conducted at a national mental health agency in The Netherlands. Main analyses comprise inter-rater reliability of checklist items, correlations to examine the relationship between checklist items and author survey (convergent validity) as well as checklist items and trial outcomes and multivariate meta-regression techniques to assess potential mechanisms of how allegiance affects trial outcomes (publication bias and RoB).Ethics and disseminationThis study has been reviewed and approved by the Scientific and Ethical Review Board (VCWE) at the Vrije Universiteit Amsterdam. Study result and advancements will also be published on the Open Science Framework. Furthermore, main findings will be disseminated through articles in international peer-reviewed open access journals. Results and recommendations will be communicated to the Cochrane Collaboration, the Campbell Collaboration and other funding agencies.