Dataset Information

Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals.

ABSTRACT: Journal policy on research data and code availability is an important part of the ongoing shift toward publishing reproducible computational science. This article extends the literature by studying journal data sharing policies by year (for both 2011 and 2012) for a referent set of 170 journals. We make a further contribution by evaluating code sharing policies, supplemental materials policies, and open access status for these 170 journals for each of 2011 and 2012. We build a predictive model of open data and code policy adoption as a function of impact factor and publisher and find higher impact journals more likely to have open data and code policies and scientific societies more likely to have open data and code policies than commercial publishers. We also find open data policies tend to lead open code policies, and we find no relationship between open data and code policies and either supplemental material policies or open access journal status. Of the journals in this study, 38% had a data policy, 22% had a code policy, and 66% had a supplemental materials policy as of June 2012. This reflects a striking one year increase of 16% in the number of data policies, a 30% increase in code policies, and a 7% increase in the number of supplemental materials policies. We introduce a new dataset to the community that categorizes data and code sharing, supplemental materials, and open access policies in 2011 and 2012 for these 170 journals.

SUBMITTER: Stodden V

PROVIDER: S-EPMC3689732 | biostudies-literature | 2013

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals.

Stodden Victoria V Guo Peixuan P Ma Zhaokun Z

PloS one 20130621 6

Journal policy on research data and code availability is an important part of the ongoing shift toward publishing reproducible computational science. This article extends the literature by studying journal data sharing policies by year (for both 2011 and 2012) for a referent set of 170 journals. We make a further contribution by evaluating code sharing policies, supplemental materials policies, and open access status for these 170 journals for each of 2011 and 2012. We build a predictive model o ...[more]

PMID: 23805293

Similar Datasets

Project description:ImportanceClinical trial data sharing holds promise for maximizing the value of clinical research. The International Committee of Medical Journal Editors (ICMJE) adopted a policy promoting data sharing in July 2018.ObjectiveTo evaluate the association of the ICMJE data sharing policy with data availability and reproducibility of main conclusions among leading surgical journals.Design, setting, and participantsThis cross-sectional study, conducted in October 2021, examined randomized clinical trials (RCTs) in 10 leading surgical journals before and after the implementation of the ICMJE data sharing policy in July 2018.ExposureImplementation of the ICMJE data sharing policy.Main outcomes and measuresTo demonstrate a pre-post increase in data availability from 5% to 25% (α = .05; β = 0.1), 65 RCTs published before and 65 RCTs published after the policy was issued were included, and their data were requested. The primary outcome was data availability (ie, the receipt of sufficient data to enable reanalysis of the primary outcome). When data sharing was available, the primary outcomes reported in the journal articles were reanalyzed to explore reproducibility. The reproducibility features of these studies were detailed.ResultsData were available for 2 of 65 RCTs (3.1%) published before the ICMJE policy and for 2 of 65 RCTs (3.1%) published after the policy was issued (odds ratio, 1.00; 95% CI, 0.07-14.19; P > .99). A data sharing statement was observed in 11 of 65 RCTs (16.9%) published after the policy vs none before the policy (risk ratio, 2.20; 95% CI, 1.81-2.68; P = .001). Data obtained for reanalysis (n = 4) were not from RCTs published with a data sharing statement. Of the 4 RCTs with available data, all of them had primary outcomes that were fully reproduced. However, discrepancies or inaccuracies that were not associated with study conclusions were identified in 3 RCTs. These concerned the number of patients included in 1 RCT, the management of missing values in another RCT, and discrepant timing for the principal outcome declared in the study registration and reported in the third RCT.Conclusions and relevanceThis cross-sectional study suggests that data sharing practices are rare in surgical journals despite the ICMJE policy and that most RCTs published in these journals lack transparency. The results of these studies may not be reproducible by external researchers.

Dataset Information

Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals.

Publications

Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets