Accuracy of Chatbots in Citing Journal Articles.
Ontology highlight
ABSTRACT:
SUBMITTER: Chen A
PROVIDER: S-EPMC10410472 | biostudies-literature | 2023 Aug
REPOSITORIES: biostudies-literature
Publications
Accuracy of Chatbots in Citing Journal Articles.
JAMA network open 20230801 8
Similar Datasets
Project description:Background. Quotations and references are an indispensable element of scientific communication. They should support what authors claim or provide important background information for readers. Studies indicate, however, that quotations not serving their purpose-quotation errors-may be prevalent. Methods. We carried out a systematic review, meta-analysis and meta-regression of quotation errors, taking account of differences between studies in error ascertainment. Results. Out of 559 studies screened we included 28 in the main analysis, and estimated major, minor and total quotation error rates of 11,9%, 95% CI [8.4, 16.6] 11.5% [8.3, 15.7], and 25.4% [19.5, 32.4]. While heterogeneity was substantial, even the lowest estimate of total quotation errors was considerable (6.7%). Indirect references accounted for less than one sixth of all quotation problems. The findings remained robust in a number of sensitivity and subgroup analyses (including risk of bias analysis) and in meta-regression. There was no indication of publication bias. Conclusions. Readers of medical journal articles should be aware of the fact that quotation errors are common. Measures against quotation errors include spot checks by editors and reviewers, correct placement of citations in the text, and declarations by authors that they have checked cited material. Future research should elucidate if and to what degree quotation errors are detrimental to scientific progress.
| S-EPMC4627914 | biostudies-literature
Project description:Enthusiasm for using Twitter as a source of data in the social sciences extends to measuring the impact of research with Twitter data being a key component in the new altmetrics approach. In this paper, we examine tweets containing links to research articles in the field of dentistry to assess the extent to which tweeting about scientific papers signifies engagement with, attention to, or consumption of scientific literature. The main goal is to better comprehend the role Twitter plays in scholarly communication and the potential value of tweet counts as traces of broader engagement with scientific literature. In particular, the pattern of tweeting to the top ten most tweeted scientific dental articles and of tweeting by accounts is examined. The ideal that tweeting about scholarly articles represents curating and informing about state-of-the-art appears not to be realized in practice. We see much presumably human tweeting almost entirely mechanical and devoid of original thought, no evidence of conversation, tweets generated by monomania, duplicate tweeting from many accounts under centralized professional management and tweets generated by bots. Some accounts exemplify the ideal, but they represent less than 10% of tweets. Therefore, any conclusions drawn from twitter data is swamped by the mechanical nature of the bulk of tweeting behavior. In light of these results, we discuss the compatibility of Twitter with the research enterprise as well as some of the financial incentives behind these patterns.
| S-EPMC5570264 | biostudies-other
Project description:BackgroundAn individual's research domain (RD) can be determined from objective publication data (e.g., medical subject headings and Medical Subject Headings (MeSH) terms) by performing social network analysis. Bibliographic coupling (such as cocitation) is a similarity metric that relies on citation analysis to determine the similarity in RD between 2 articles. This study compared RD consistency between articles as well as their cited references and citing articles (ARCs).MethodsA total of 1388 abstracts were downloaded from PubMed and authored by 3 productive authors. Based on the top 3 clusters in social network analysis, similarity in RD was observed by comparing their consistency using the major MeSH terms in author articles, cited references and citing articles (ARC). Impact beam plots with La indices were drawn and compared for each of the 3 authors.ResultsSung-Ho Jang (South Korea), Chia-Hung Kao (Taiwan), and Chin-Hsiao Tseng (Taiwan) published 445, 780, and 163 articles, respectively. Dr Jang's RD is physiology, and Dr Kao and Dr Tseng's RDs are epidemiology. We confirmed the consistency of the RD terms by comparing the major MeSH terms in the ARC. Their La indexes were 5, 5, and 6, where a higher value indicates more extraordinary research achievement.ConclusionRD consistency was confirmed by comparing the main MeSH terms in ARC. The 3 approaches of RD determination (based on author articles, the La index, and the impact beam plots) were recommended for bibliographical studies in the future.
| S-EPMC9646507 | biostudies-literature
Project description:This data article presents a corpus (i.e. a selection of a big number of words in an electronic form) and a concordancer (i.e. a tool to show the word in its context of use) of academic journal articles. As the title suggests, the data were collected from research articles published in academic journals. The corpus contains 5,686,428 words selected from 895 journal articles published by Elsevier in 2011-2015. The corpus is classified into four subject areas: Health sciences, Life sciences, Physical Sciences, and Social Sciences, following the classifications of Scopus, which is the largest abstract and citation database of peer-reviewed scientific journals, books and conference proceedings. To ease the access and utilization of the corpus, a program to produce the key word in context (KWIC) and word frequency was created and placed on the website: corpus.kwary.net. The corpus is a valuable resource for researchers, teachers, and translators working on academic English.
| S-EPMC5694964 | biostudies-literature
Project description:This article describes a systematic analysis of the relationship between empirical data and theoretical conclusions for a set of experimental psychology articles published in the journal Science between 2005-2012. When the success rate of a set of empirical studies is much higher than would be expected relative to the experiments' reported effects and sample sizes, it suggests that null findings have been suppressed, that the experiments or analyses were inappropriate, or that the theory does not properly follow from the data. The analyses herein indicate such excess success for 83% (15 out of 18) of the articles in Science that report four or more studies and contain sufficient information for the analysis. This result suggests a systematic pattern of excess success among psychology articles in the journal Science.
| S-EPMC4256411 | biostudies-literature
Project description:ObjectiveResearch on publication trends in journal articles on sleep disorders (SDs) and the associated methodologies by using text mining has been limited. The present study involved text mining for terms to determine the publication trends in sleep-related journal articles published during 2000-2013 and to identify associations between SD and methodology terms as well as conducting statistical analyses of the text mining findings.MethodsSD and methodology terms were extracted from 3,720 sleep-related journal articles in the PubMed database by using MetaMap. The extracted data set was analyzed using hierarchical cluster analyses and adjusted logistic regression models to investigate publication trends and associations between SD and methodology terms.ResultsMetaMap had a text mining precision, recall, and false positive rate of 0.70, 0.77, and 11.51%, respectively. The most common SD term was breathing-related sleep disorder, whereas narcolepsy was the least common. Cluster analyses showed similar methodology clusters for each SD term, except narcolepsy. The logistic regression models showed an increasing prevalence of insomnia, parasomnia, and other sleep disorders but a decreasing prevalence of breathing-related sleep disorder during 2000-2013. Different SD terms were positively associated with different methodology terms regarding research design terms, measure terms, and analysis terms.ConclusionInsomnia-, parasomnia-, and other sleep disorder-related articles showed an increasing publication trend, whereas those related to breathing-related sleep disorder showed a decreasing trend. Furthermore, experimental studies more commonly focused on hypersomnia and other SDs and less commonly on insomnia, breathing-related sleep disorder, narcolepsy, and parasomnia. Thus, text mining may facilitate the exploration of the publication trends in SDs and the associated methodologies.
| S-EPMC4874549 | biostudies-literature
Project description:ImportanceWomen are less likely to be promoted and hold leadership positions in academic medicine. How often academic articles are cited is a key measure of scholarly impact and frequently assessed for professional advancement; however, it is unknown whether peer-reviewed articles written by men and women are cited differently.ObjectiveTo evaluate whether academic articles from high-impact medical journals written by men and women are cited differently.Design, setting, and participantsIn this cross-sectional study of all original research and commentary articles from 5 high-impact medical journals (Annals of Internal Medicine, British Medical Journal, JAMA, JAMA Internal Medicine, and The New England Journal of Medicine) from 2015 to 2018, the gender of the primary and senior authors of each article were identified using an online database, and the number of times each article has been cited was identified using Web of Science. The number of citations by primary and senior author gender were then compared. Data were analyzed from July 2020 to April 2021.ExposuresPrimary and senior authors' genders.Main outcomes and measuresNumber of citations per article.ResultsAmong 5554 articles, women wrote 1975 (35.6%) as primary author and 1273 of 4940 (25.8%) as senior author. Original research articles written by women as primary authors had fewer median (interquartile range) citations than articles written by men as primary authors (36 [17-82] citations vs 54 [22-141] citations; P < .001) and senior authors (37 [17-93] citations vs 51 [20-128] citations; P < .001). Articles written by women as both primary and senior authors had approximately half as many median (interquartile range) citations as those authored by men as both primary and senior authors (33 [15-68] citations vs 59 [23-149] citations; P < .001). Differences in citations remained in each year of the study and were less pronounced among commentary articles.Conclusions and relevanceIn this study, articles written by women in high-impact medical journals had fewer citations than those written by men, particularly when women wrote together as primary and senior authors. These differences may have important consequences for the professional success of women and achieving gender equity in academic medicine.
| S-EPMC8254129 | biostudies-literature
Project description:Primary scientific literature can be difficult to navigate for anyone unfamiliar with its foreign, formal structure. We sought to create a fun, easy learning tool to help familiarize students of all ages with the structure of a scientific article. Our main learning objective was for the student to realize that science writing is formulaic-that specific information is found in predictable locations within an article-and that, with an understanding of the formula, anyone can comfortably navigate any journal article and accurately predict what to expect to find in each section. To this end, we designed a Journal Article Scavenger Hunt that requires the user to find and identify a series of commonplace features of a primary research article. The scavenger hunt activity is quick and easy to implement, and is adaptable to various ages and settings, including the classroom, lab, and at outreach events. The questions in the scavenger hunt can be scaled in difficulty and specificity to suit the instructor's needs. Over many years of using this activity, we have received positive feedback from students of all ages, from elementary school students to lay adult-learners as well as science teachers themselves. By making the unknown seem predictable and approachable, the scavenger hunt helps a variety of audiences feel more comfortable with science and more confident in their ability to engage directly with the scientific literature. Journal of Microbiology & Biology Education.
| S-EPMC4798794 | biostudies-literature
Project description:BackgroundTo practice evidence-based medicine, physicians should have a solid understanding of fundamental epidemiological and biostatistical concepts. Research suggests that only a minority of physicians have such an understanding of biostatistics.ObjectiveTo collect pilot data on a novel biostatistical educational tool, a hyperlink-embedded journal article, which is aimed at improving knowledge in biostatistics.MethodsForty-four physicians-in-training participated in this pilot study. Participants completed a pretest consisting of 5 questions about biostatistical terms that would be encountered in the article. They were randomized to either an unmodified journal article or to the same article with hyperlinked biostatistical terms. All participants then completed a posttest that was identical to the pretest.ResultsHaving access to hyperlinked information had a positive association with the number of improved test answers (P = .05). Use of hyperlinks varied, and were seemingly dependent on user comfort with terms; well-understood definitions ("average") were clicked on a few times (5.5% of participants), whereas more obscure method terms ("Lexis diagram") were clicked on by 94% of participants. While only 42% of participants stated they would have looked up definitions of the biostatistical terms if they had not been provided in the hyperlinked article, 94% of participants identified the hyperlink tool as something they would use if readily available to them when reading journal articles.ConclusionsResults of this pilot study of a novel educational intervention suggest that embedded hyperlinks within journal articles may be a useful tool to teach biostatistical terms to physicians.
| S-EPMC4675424 | biostudies-literature
Project description:ImportanceCitation analysis is a bibliometric method that uses citation rates to evaluate research performance. This type of analysis can identify the articles that have shaped the modern history of obstetrics and gynecology (OBGYN).ObjectivesTo identify and characterize top-cited OBGYN articles in the Institute for Scientific Information Web of Science's Science Citation Index Expanded and to compare top-cited OBGYN articles published in specialty OBGYN journals with those published in nonspecialty journals.Design, setting, and participantsCross-sectional bibliometric analysis of top-cited articles that were indexed in the Science Citation Index Expanded from 1980 to 2018. The Science Citation Index Expanded was queried using search terms from the American Board of Obstetrics and Gynecology's 2018 certifying examination topics list. The top 100 articles from all journals and the top 100 articles from OBGYN journals were evaluated for specific characteristics. Data were analyzed in March 2019.Main outcomes and measuresThe articles were characterized by citation number, publication year, topic, study design, and authorship. After excluding articles that featured on both lists, top-cited articles were compared.ResultsThe query identified 3 767 874 articles, of which 278 846 (7.4%) were published in OBGYN journals. The top-cited article was published by Rossouw and colleagues in JAMA (2002). Top-cited articles published in nonspecialty journals were more frequently cited than those in OBGYN journals (median [interquartile range], 1738 [1490-2077] citations vs 666 [580-843] citations, respectively; P < .001) and were more likely to be randomized trials (25.0% vs 2.2%, respectively; difference, 22.8%; 95% CI, 13.5%-32.2%; P < .001). Whereas articles from nonspecialty journals focused on broad topics like osteoporosis, articles from OBGYN journal focused on topics like preeclampsia and endometriosis.Conclusions and relevanceThis study found substantial differences between top-cited OBGYN articles published in nonspecialty vs OBGYN journals. These differences may reflect the different goals of the journals, which work together to ensure optimal dissemination of impactful articles.
| S-EPMC6991228 | biostudies-literature