Dataset Information

An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017).

ABSTRACT: Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

SUBMITTER: Hardwicke TE

PROVIDER: S-EPMC7062098 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000-2014, the biomedical literature largely lacked transparency in important dimensions. Here, we surveyed a random sample of 149 biomedical articles published between 2015 and 2017 and determined the proportion reporting sources of public and/or private funding and conflicts of interests, sharing protocols and raw data, and undergoing rigorous independent replication and reproducibility checks. We also investigated what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. The majority of the 149 studies disclosed some information regarding funding (103, 69.1% [95% confidence interval, 61.0% to 76.3%]) or conflicts of interest (97, 65.1% [56.8% to 72.6%]). Among the 104 articles with empirical data in which protocols or data sharing would be pertinent, 19 (18.3% [11.6% to 27.3%]) discussed publicly available data; only one (1.0% [0.1% to 6.0%]) included a link to a full study protocol. Among the 97 articles in which replication in studies with different data would be pertinent, there were five replication efforts (5.2% [1.9% to 12.2%]). Although clinical trial identification numbers and funding details were often provided on PubMed, only two of the articles without a full text article in PubMed Central that discussed publicly available data at the full text level also contained information related to data sharing on PubMed; none had a conflicts of interest statement on PubMed. Our evaluation suggests that although there have been improvements over the last few years in certain key indicators of reproducibility and transparency, opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.

Project description:BACKGROUND:Strengthening research capacity in low-and-middle-income countries is essential to drive socioeconomic development and to achieve the Sustainable Development Goals. Understanding strengths and weaknesses in institutions' research capacity can guide effective targeting of investments and resources. This study assessed the capacity of institutions undertaking research in natural science topics in Africa to identify priority capacity gaps for future investment. METHODS:Assessments were conducted in eight African institutions that were partners in a UK-Africa programme to strengthen research capacity in renewable energy, soil-related science, and water and sanitation. Assessments involved eighty-six interviews and three focus group discussions to identify institutions' research capacity strengths and gaps against an evidence-informed benchmark. Use of the same interview guides and data collection processes across all institutions meant that findings could be compared. RESULTS:Common research capacity gaps were: lack of, or poorly maintained, equipment; unreliable, slow procurement systems; insufficient opportunities for developing the skills of research support staff such as administrators and technicians; dysfunctional institutional email communication systems; insufficient focus on the development of 'soft' researcher skills such as ethics, academic writing and, in non-Anglophone countries, English language. Programme strengths were the South-South and South-North partnerships for sharing and cascading expertise and resources, joint writing of proposals and publications, and improved individual and institutional visibility. CONCLUSION:There were many similarities in research capacity gaps irrespective of the institutions' natural sciences research focus, and these were similar to those reported in the health sector. Common capacity needs are improving the skills of technicians and administrators to support research activities, soft skills training for researchers, and more effective pan-institutional e-communication systems. These could be strategic investment targets for the joint efforts of national governments and international organisations that fund programmes for strengthening research capacity in low- and middle-income countries.

Dataset Information

An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets