Dataset Information

The effectiveness of journals as arbiters of scientific impact.

ABSTRACT: Academic publishers purport to be arbiters of knowledge, aiming to publish studies that advance the frontiers of their research domain. Yet the effectiveness of journal editors at identifying novel and important research is generally unknown, in part because of the confidential nature of the editorial and peer review process. Using questionnaires, we evaluated the degree to which journals are effective arbiters of scientific impact on the domain of Ecology, quantified by three key criteria. First, journals discriminated against low-impact manuscripts: The probability of rejection increased as the number of citations gained by the published paper decreased. Second, journals were more likely to publish high-impact manuscripts (those that obtained citations in 90th percentile for their journal) than run-of-the-mill manuscripts; editors were only 23% and 41% as likely to reject an eventual high-impact paper (pre- versus postreview rejection) compared to a run-of-the-mill paper. Third, editors did occasionally reject papers that went on to be highly cited. Error rates were low, however: Only 3.8% of rejected papers gained more citations than the median article in the journal that rejected them, and only 9.2% of rejected manuscripts went on to be high-impact papers in the (generally lower impact factor) publishing journal. The effectiveness of scientific arbitration increased with journal prominence, although some highly prominent journals were no more effective than much less prominent ones. We conclude that the academic publishing system, founded on peer review, appropriately recognizes the significance of research contained in manuscripts, as measured by the number of citations that manuscripts obtain after publication, even though some errors are made. We therefore recommend that authors reduce publication delays by choosing journals appropriate to the significance of their research.

SUBMITTER: Paine CET

PROVIDER: S-EPMC6202707 | biostudies-literature | 2018 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The effectiveness of journals as arbiters of scientific impact.

Paine C E Timothy CET Fox Charles W CW

Ecology and evolution 20180912 19

Academic publishers purport to be arbiters of knowledge, aiming to publish studies that advance the frontiers of their research domain. Yet the effectiveness of journal editors at identifying novel and important research is generally unknown, in part because of the confidential nature of the editorial and peer review process. Using questionnaires, we evaluated the degree to which journals are effective arbiters of scientific impact on the domain of Ecology, quantified by three key criteria. Firs ...[more]

PMID: 30386557

Similar Datasets

Project description:In recent decades, we have observed an intensification of science, technology and innovation activities in Brazil. The increase in production of scientific papers indexed in international databases, however, has not been accompanied by an equivalent increase in the impact of publications. This paper presents a methodology for analyzing production and the impact of certain research areas in Brazil related to two aspects: the origin of the journals (national or foreign) and international collaboration. These two variables were selected for being of particular importance in understanding the context of scientific production and communication in countries with emerging economies. The sample consisted of papers written by Brazilian researchers in 19 subfields of knowledge published from 2002 to 2011, totaling 85,082 papers. To calculate the impact, we adopted a normalized indicator called the relative subfield citedness (Rw) using a window of 5 years to obtain measurements evaluated in 2 different years: 2007 and 2012. The data on papers and citations were collected from the Web of Science database. From the results, we note that most of the subfields have presented, from one quinquennium to another, improved performance in the world production rankings. Regarding publication in national and foreign journals, we observed a trend in the distribution maintenance of production of the subfields based on the origin of the journal. Specifically, for impact, we identified a lower Rw pattern for Brazilian papers when they were published in national journals in all subfields. When Brazilian products are published in foreign journals, we observed a higher impact for those papers, even surpassing the average global impact in some subfields. For international collaboration, we analyzed the percentage of participation of foreign researchers and the connection between collaboration and the impact of papers, especially emphasizing the distinction of hyperauthorship papers in terms of production and impact.

Project description:The aim of this study is to reveal a robust correlation between the amount of attention international journalism devotes to scientific papers and the amount of attention scientific journals devote to the respective topics. Using a Mainstream-Media-Score (MSM) ≥ 100 (which we regard as an indicator for news media attention) from the altmetrics provider Altmetric, we link 983 research articles with 185,166 thematically similar articles from the PubMed database (which we use to operationalize attention from scientific journals). The method we use is to test whether there is a concomitant increase in scientific attention after a research article has received popular media coverage. To do so, we compare the quotient of the number of thematically similar articles published in scientific journals during the period before and after the publication of an MSM ≥ 100 article. Our main result shows that in 59 percent of cases, more thematically similar articles were published in scientific journals after a scientific paper received noteworthy news media coverage than before (p < 0.01). In this context, we neither found significant differences between various types of scientific journal (p = 0.3) nor between scientific papers that were originally published in renowned opinion-leading journals or in less renowned, non-opinion-leading journals (p = 0.1). Our findings indicate a robust correlation between the choice of topics in the mass media and in research. However, our study cannot clarify whether this correlation occurs because researchers and/or scientific journals are oriented towards public relevance (publicity effect) or whether the correlation is due to the parallelism of relevance attributions in quality journalism and research (earmark hypothesis). We infer that topics of social relevance are (more) likely to be picked up by popular media as well as by scientific journals. Altogether, our study contributes new empirical findings to the relationship between topic selection in journalism and in research.

Project description:BackgroundPublishing identifiable patient data in scientific journals may jeopardize patient privacy and confidentiality if best ethical practices are not followed. Current journal practices show considerable diversity in the publication of identifiable patient photographs, and different stakeholders may have different opinions of and practices in publishing patient photographs.ObjectiveThis scoping review aimed to identify existing evidence and map knowledge gaps in medical research on the policies and practices of publishing identifiable photographs in scientific articles.MethodsWe performed a comprehensive search of the Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, CINAHL with Full Text, Database of Abstracts of Reviews of Effects, Ovid MEDLINE, and Scopus. The Open Science Framework, PROSPERO, BASE, Google Scholar, OpenGrey, ClinicalTrials.gov, the Campbell Collaboration Library, and Science.gov were also searched.ResultsAfter screening the initial 15,949 titles and abstracts, 98 (0.61%) publications were assessed for eligibility at the full-text level, and 30 (0.19%) publications were included in this review. The studies were published between 1994 and 2020; most had a cross-sectional design and were published in journals covering different medical disciplines. We identified 3 main topics. The first included ethical aspects of the use of facial photographs in publications. In different clinical settings, the consent process was not conducted properly, and health professionals did not recognize the importance of obtaining written patient consent for taking and using patient medical photographs. They often considered verbal consent sufficient or even used the photographs without consent. The second topic included studies that investigated the practices and use of medical photography in publishing. Both patients and doctors asked for confidential storage and maintenance of medical photographs. Patients preferred to be photographed by their physicians using an institutional camera and preferred nonidentifiable medical photographs not only for publication but also in general. Conventional methods of deidentification of facial photographs concealing the eye area were recognized as unsuccessful in protecting patient privacy. The third topic emerged from studies investigating medical photography in journal articles. These studies showed great diversity in publishing practices regarding consent for publication of medical photographs. Journal policies regarding the consent process and consent forms were insufficient, and existing ethical professional guidelines were not fully implemented in actual practices. Patients' photographs from open-access medical journals were found on public web-based platforms.ConclusionsThis scoping review showed a diversity of practices in publishing identifiable patient photographs and an unsatisfactory level of knowledge of this issue among different stakeholders despite existing standards. Emerging issues include the availability of patients' photographs from open-access journals or preprints in the digital environment. There is a need to improve standards and processes to obtain proper consent to fully protect the privacy of patients in published articles.

Project description:BackgroundIt is not clear which research misconduct policies are adopted by biomedical journals. This study assessed the prevalence and content policies of the most influential biomedical journals on misconduct and procedures for handling and responding to allegations of misconduct.MethodsWe conducted a cross-sectional study of misconduct policies of 399 high-impact biomedical journals in 27 biomedical categories of the Journal Citation Reports in December 2011. Journal websites were reviewed for information relevant to misconduct policies.ResultsOf 399 journals, 140 (35.1%) provided explicit definitions of misconduct. Falsification was explicitly mentioned by 113 (28.3%) journals, fabrication by 104 (26.1%), plagiarism by 224 (56.1%), duplication by 242 (60.7%) and image manipulation by 154 (38.6%). Procedures for responding to misconduct were described in 179 (44.9%) websites, including retraction, (30.8%) and expression of concern (16.3%). Plagiarism-checking services were used by 112 (28.1%) journals. The prevalences of all types of misconduct policies were higher in journals that endorsed any policy from editors' associations, Office of Research Integrity or professional societies compared to those that did not state adherence to these policy-producing bodies. Elsevier and Wiley-Blackwell had the most journals included (22.6% and 14.8%, respectively), with Wiley journals having greater a prevalence of misconduct definition and policies on falsification, fabrication and expression of concern and Elsevier of plagiarism-checking services.ConclusionsOnly a third of top-ranking peer-reviewed journals had publicly-available definitions of misconduct and less than a half described procedures for handling allegations of misconduct. As endorsement of international policies from policy-producing bodies was positively associated with implementation of policies and procedures, journals and their publishers should standardize their policies globally in order to increase public trust in the integrity of the published record in biomedicine.

Project description:Background In the period 2003-2008, the regulatory authorities issued several warnings restricting the use of selective serotonin re-uptake inhibitors (SSRIs) in paediatrics, in reaction to safety concerns regarding the risk of suicidality. In this study, the SSRIs and suicidality controversy serves as a template to analyse the long-term publication trends regarding the benefit/risk profile of medications. The aim is to ascertain differences (in terms of numbers, categories and timing) between negative and positive newspaper and journal articles on SSRIs and suicidality and to ascertain correlations between changes in the reports and regulatory warnings. Methods A systematic review of scientific articles (Embase) and the Netherlands (NL) and the UK newspapers (LexisNexis) was performed between 2000 and 2010. Categorisation was done by 'effect' (related treatment effect), 'type of article' and 'age group'. The articles' positive-to-negative effect ratio was determined. Differences in distribution of effect categories were analysed across sources, type of article and age group using the Mann-Whitney (two subgroups) or Kruskal-Wallis test (three or more). Findings In total, 1141 articles were categorised: 352 scientific, 224 Dutch and 565 British newspaper articles. Scientific articles were predominantly on research and were positive, whereas newspaper articles were negative (ratios=3.50-scientific, 0.69-NL and 0.94-UK; p<0.001). Articles on paediatrics were less positive in scientific journals and more negative in newspapers (ratios=2.29-scientific, 0.26-NL and 0.20-UK; p<0.001), while articles on adults were positive overall (ratios=10.0-scientific, 1.06-NL and 1.70-UK; p<0.001). In addition, negative-effect reporting trends were exacerbated following regulatory warnings and were generally opinion articles, both in scientific journals and in newspapers (2003/2004 and after 2007). Interpretation The authors found a positive publication tendency inherent in journal research articles. This apparent positive publication bias present in scientific journals, however, does not seem to prevent the dissemination of 'bad' news about medications. The negative tendency present in Dutch and British newspapers was perceivable in the paediatrics group and during the warnings, indicating that national news media have informed the public about this international drug safety controversy on time.

Dataset Information

The effectiveness of journals as arbiters of scientific impact.

Publications

The effectiveness of journals as arbiters of scientific impact.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets