Excess success for psychology articles in the journal science.
Ontology highlight
ABSTRACT:
SUBMITTER: Francis G
PROVIDER: S-EPMC4256411 | biostudies-literature | 2014
REPOSITORIES: biostudies-literature
Publications
Excess success for psychology articles in the journal science.
Francis Gregory G Tanzman Jay J Matthews William J WJ
PloS one 20141204 12
This article describes a systematic analysis of the relationship between empirical data and theoretical conclusions for a set of experimental psychology articles published in the journal Science between 2005-2012. When the success rate of a set of empirical studies is much higher than would be expected relative to the experiments' reported effects and sample sizes, it suggests that null findings have been suppressed, that the experiments or analyses were inappropriate, or that the theory does no ...[more]
Similar Datasets
Project description:Twenty-five years of research has explored the object-based attention effect using the two-rectangles paradigm and closely related paradigms. While reading this literature, we noticed statistical attributes that are sometimes related to questionable research practices, which can undermine the reported conclusions. To quantify these attributes, we applied the Test for Excess Success (TES) individually to 37 articles that investigate various properties of object-based attention and comprise four or more experiments. A TES analysis estimates the probability that a direct replication of the experiments in a given article with the same sample sizes would have the same success (or better) as the original article. If the probability is low, then readers should be skeptical about the conclusions that are based on those experimental results. We find that 19 of the 37 analyzed articles (51%) seem too good to be true in that they have a replication probability below 0.1. In a new large sample study, we do find evidence for the basic object-based attention effect in the two-rectangles paradigm, which this literature builds on. A power analysis using this data shows that commonly used sample sizes in studies that investigate properties of object-based attention with the two-rectangles paradigm are, in fact, much too small to reliably detect even the basic effect.
| S-EPMC8887804 | biostudies-literature
Project description:BackgroundWhile environmental research addresses scientific questions of possible societal relevance, it is unclear to what degree research focuses on environmental chemicals in need of documentation for risk assessment purposes.MethodsIn a bibliometric analysis, we used SciFinder to extract Chemical Abstract Service (CAS) numbers for chemicals addressed by publications in the 78 major environmental science journals during 2000-2009. The Web of Science was used to conduct title searches to determine long-term trends for prominent substances and substances considered in need of research attention.ResultsThe 119,636 journal articles found had 760,056 CAS number links during 2000-2009. The top-20 environmental chemicals consisted of metals, (chlorinated) biphenyls, polyaromatic hydrocarbons, benzene, and ethanol and contributed 12% toward the total number of links- Each of the top-20 substances was covered by 2,000-10,000 articles during the decade. The numbers for the 10-year period were similar to the total numbers of pre-2000 articles on the same chemicals. However, substances considered a high priority from a regulatory viewpoint, due to lack of documentation, showed very low publication rates. The persistence in the scientific literature of the top-20 chemicals was only weakly related to their publication in journals with a high impact factor, but some substances achieved high citation rates.ConclusionsThe persistence of some environmental chemicals in the scientific literature may be due to a 'Matthew' principle of maintaining prominence for the very reason of having been well researched. Such bias detracts from the societal needs for documentation on less well known environmental hazards, and it may also impact negatively on the potentials for innovation and discovery in research.
| S-EPMC3229577 | biostudies-literature
Project description: Not available
| S-EPMC10410472 | biostudies-literature
Project description:Enthusiasm for using Twitter as a source of data in the social sciences extends to measuring the impact of research with Twitter data being a key component in the new altmetrics approach. In this paper, we examine tweets containing links to research articles in the field of dentistry to assess the extent to which tweeting about scientific papers signifies engagement with, attention to, or consumption of scientific literature. The main goal is to better comprehend the role Twitter plays in scholarly communication and the potential value of tweet counts as traces of broader engagement with scientific literature. In particular, the pattern of tweeting to the top ten most tweeted scientific dental articles and of tweeting by accounts is examined. The ideal that tweeting about scholarly articles represents curating and informing about state-of-the-art appears not to be realized in practice. We see much presumably human tweeting almost entirely mechanical and devoid of original thought, no evidence of conversation, tweets generated by monomania, duplicate tweeting from many accounts under centralized professional management and tweets generated by bots. Some accounts exemplify the ideal, but they represent less than 10% of tweets. Therefore, any conclusions drawn from twitter data is swamped by the mechanical nature of the bulk of tweeting behavior. In light of these results, we discuss the compatibility of Twitter with the research enterprise as well as some of the financial incentives behind these patterns.
| S-EPMC5570264 | biostudies-other
Project description:ObjectiveResearch on publication trends in journal articles on sleep disorders (SDs) and the associated methodologies by using text mining has been limited. The present study involved text mining for terms to determine the publication trends in sleep-related journal articles published during 2000-2013 and to identify associations between SD and methodology terms as well as conducting statistical analyses of the text mining findings.MethodsSD and methodology terms were extracted from 3,720 sleep-related journal articles in the PubMed database by using MetaMap. The extracted data set was analyzed using hierarchical cluster analyses and adjusted logistic regression models to investigate publication trends and associations between SD and methodology terms.ResultsMetaMap had a text mining precision, recall, and false positive rate of 0.70, 0.77, and 11.51%, respectively. The most common SD term was breathing-related sleep disorder, whereas narcolepsy was the least common. Cluster analyses showed similar methodology clusters for each SD term, except narcolepsy. The logistic regression models showed an increasing prevalence of insomnia, parasomnia, and other sleep disorders but a decreasing prevalence of breathing-related sleep disorder during 2000-2013. Different SD terms were positively associated with different methodology terms regarding research design terms, measure terms, and analysis terms.ConclusionInsomnia-, parasomnia-, and other sleep disorder-related articles showed an increasing publication trend, whereas those related to breathing-related sleep disorder showed a decreasing trend. Furthermore, experimental studies more commonly focused on hypersomnia and other SDs and less commonly on insomnia, breathing-related sleep disorder, narcolepsy, and parasomnia. Thus, text mining may facilitate the exploration of the publication trends in SDs and the associated methodologies.
| S-EPMC4874549 | biostudies-literature
Project description:This data article presents a corpus (i.e. a selection of a big number of words in an electronic form) and a concordancer (i.e. a tool to show the word in its context of use) of academic journal articles. As the title suggests, the data were collected from research articles published in academic journals. The corpus contains 5,686,428 words selected from 895 journal articles published by Elsevier in 2011-2015. The corpus is classified into four subject areas: Health sciences, Life sciences, Physical Sciences, and Social Sciences, following the classifications of Scopus, which is the largest abstract and citation database of peer-reviewed scientific journals, books and conference proceedings. To ease the access and utilization of the corpus, a program to produce the key word in context (KWIC) and word frequency was created and placed on the website: corpus.kwary.net. The corpus is a valuable resource for researchers, teachers, and translators working on academic English.
| S-EPMC5694964 | biostudies-literature
Project description:ImportanceWomen are less likely to be promoted and hold leadership positions in academic medicine. How often academic articles are cited is a key measure of scholarly impact and frequently assessed for professional advancement; however, it is unknown whether peer-reviewed articles written by men and women are cited differently.ObjectiveTo evaluate whether academic articles from high-impact medical journals written by men and women are cited differently.Design, setting, and participantsIn this cross-sectional study of all original research and commentary articles from 5 high-impact medical journals (Annals of Internal Medicine, British Medical Journal, JAMA, JAMA Internal Medicine, and The New England Journal of Medicine) from 2015 to 2018, the gender of the primary and senior authors of each article were identified using an online database, and the number of times each article has been cited was identified using Web of Science. The number of citations by primary and senior author gender were then compared. Data were analyzed from July 2020 to April 2021.ExposuresPrimary and senior authors' genders.Main outcomes and measuresNumber of citations per article.ResultsAmong 5554 articles, women wrote 1975 (35.6%) as primary author and 1273 of 4940 (25.8%) as senior author. Original research articles written by women as primary authors had fewer median (interquartile range) citations than articles written by men as primary authors (36 [17-82] citations vs 54 [22-141] citations; P < .001) and senior authors (37 [17-93] citations vs 51 [20-128] citations; P < .001). Articles written by women as both primary and senior authors had approximately half as many median (interquartile range) citations as those authored by men as both primary and senior authors (33 [15-68] citations vs 59 [23-149] citations; P < .001). Differences in citations remained in each year of the study and were less pronounced among commentary articles.Conclusions and relevanceIn this study, articles written by women in high-impact medical journals had fewer citations than those written by men, particularly when women wrote together as primary and senior authors. These differences may have important consequences for the professional success of women and achieving gender equity in academic medicine.
| S-EPMC8254129 | biostudies-literature
Project description:To practice evidence-based medicine, physicians should have a solid understanding of fundamental epidemiological and biostatistical concepts. Research suggests that only a minority of physicians have such an understanding of biostatistics.To collect pilot data on a novel biostatistical educational tool, a hyperlink-embedded journal article, which is aimed at improving knowledge in biostatistics.Forty-four physicians-in-training participated in this pilot study. Participants completed a pretest consisting of 5 questions about biostatistical terms that would be encountered in the article. They were randomized to either an unmodified journal article or to the same article with hyperlinked biostatistical terms. All participants then completed a posttest that was identical to the pretest.Having access to hyperlinked information had a positive association with the number of improved test answers (P = .05). Use of hyperlinks varied, and were seemingly dependent on user comfort with terms; well-understood definitions ("average") were clicked on a few times (5.5% of participants), whereas more obscure method terms ("Lexis diagram") were clicked on by 94% of participants. While only 42% of participants stated they would have looked up definitions of the biostatistical terms if they had not been provided in the hyperlinked article, 94% of participants identified the hyperlink tool as something they would use if readily available to them when reading journal articles.Results of this pilot study of a novel educational intervention suggest that embedded hyperlinks within journal articles may be a useful tool to teach biostatistical terms to physicians.
| S-EPMC4675424 | biostudies-literature
Project description:Primary scientific literature can be difficult to navigate for anyone unfamiliar with its foreign, formal structure. We sought to create a fun, easy learning tool to help familiarize students of all ages with the structure of a scientific article. Our main learning objective was for the student to realize that science writing is formulaic-that specific information is found in predictable locations within an article-and that, with an understanding of the formula, anyone can comfortably navigate any journal article and accurately predict what to expect to find in each section. To this end, we designed a Journal Article Scavenger Hunt that requires the user to find and identify a series of commonplace features of a primary research article. The scavenger hunt activity is quick and easy to implement, and is adaptable to various ages and settings, including the classroom, lab, and at outreach events. The questions in the scavenger hunt can be scaled in difficulty and specificity to suit the instructor's needs. Over many years of using this activity, we have received positive feedback from students of all ages, from elementary school students to lay adult-learners as well as science teachers themselves. By making the unknown seem predictable and approachable, the scavenger hunt helps a variety of audiences feel more comfortable with science and more confident in their ability to engage directly with the scientific literature. Journal of Microbiology & Biology Education.
| S-EPMC4798794 | biostudies-literature
Project description:In this study, we conducted a citation network analysis of the Journal of Counseling Psychology (JCP) to elucidate the scope, evolution, and interconnections of JCP publications as reflected in how authors use (i.e., cite) these publications. We used CitNetExplorer to analyze a network of 4,718 JCP publications and 16,959 citation links. The analysis yielded 19 clusters in JCP's citation network. The most dominant facet of the citation network focused on counseling, counseling process, and counseling outcome. The clusters in this facet shifted from an early focus on perceptions of counselors to continuing examinations of what happens in counseling. Another dominant facet comprised clusters on vocational psychology, shifting from an earlier focus on vocational choice and interest, to career counseling and decision making, to vocational and academic development processes and predictors. These major facets converged toward a continuing cluster focused on methodology and analysis along with race, gender, sexuality, and other diversity scholarship. This suggests that diversity-focused publications in JCP often employ and cite methodological and analytic advances. The results also reveal discontinued areas of scholarship in JCP that are ripe for revisiting and rebuilding in new directions (e.g., anger and social justice activism; clinical judgment and artificial intelligence). The results suggest that a promising next step in the evolution of JCP would be for authors to engage with and cite diversity scholarship as central to "general" domains of JCP scholarship. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
| S-EPMC10898226 | biostudies-literature