Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations.
Ontology highlight
ABSTRACT:
SUBMITTER: McKiernan EC
PROVIDER: S-EPMC6668985 | biostudies-literature | 2019 Jul
REPOSITORIES: biostudies-literature
Publications
Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations.
McKiernan Erin C EC Schimanski Lesley A LA Muñoz Nieves Carol C Matthias Lisa L Niles Meredith T MT Alperin Juan P JP
eLife 20190731
We analyzed how often and in what ways the Journal Impact Factor (JIF) is currently used in review, promotion, and tenure (RPT) documents of a representative sample of universities from the United States and Canada. 40% of research-intensive institutions and 18% of master's institutions mentioned the JIF, or closely related terms. Of the institutions that mentioned the JIF, 87% supported its use in at least one of their RPT documents, 13% expressed caution about its use, and none heavily critici ...[more]
Similar Datasets
Project description:Review, promotion, and tenure (RPT) processes at universities typically assess candidates along three dimensions: research, teaching, and service. In recent years, some have argued for the inclusion of a controversial fourth criterion: collegiality. While collegiality plays a role in the morale and effectiveness of academic departments, it is amorphic and difficult to assess, and could be misused to stifle dissent or enforce homogeneity. Despite this, some institutions have opted to include this additional element in their RPT documents and processes, but it is unknown the extent of this practice and how it varies across institution type and disciplinary units. This study is based on two sets of data: survey data collected as part of a project that explored the publishing decisions of faculty and how these related to perceived importance in RPT processes, and 864 RPT documents collected from 129 universities from the United States and Canada. We analysed these RPT documents to determine the degree to which collegiality and related terms are mentioned, if they are defined, and if and how they may be assessed during the RPT process. Results show that when collegiality and related terms appear in these documents they are most often just briefly mentioned. It is less common for collegiality and related terms to be defined or assessed in RPT documents. Although the terms are mentioned across all types of institutions, there is a statistically significant difference in how prevalent they are at each. Collegiality is more commonly mentioned in the documents of doctoral research-focused universities (60%), than of master's universities and colleges (31%) or baccalaureate colleges (15%). Results from the accompanying survey of faculty also support this finding: individuals from R-Types were more likely to perceive collegiality to be a factor in their RPT processes. We conclude that collegiality likely plays an important role in RPT processes, whether it is explicitly acknowledged in policies and guidelines or not, and point to several strategies in how it might be best incorporated in the assessment of academic careers.
| S-EPMC8986017 | biostudies-literature
Project description:Review, promotion, and tenure (RPT) processes significantly affect how faculty direct their own career and scholarly progression. Although RPT practices vary between and within institutions, and affect various disciplines, ranks, institution types, genders, and ethnicity in different ways, some consistent themes emerge when investigating what faculty would like to change about RPT. For instance, over the last few decades, RPT processes have generally increased the value placed on research, at the expense of teaching and service, which often results in an incongruity between how faculty actually spend their time vs. what is considered in their evaluation. Another issue relates to publication practices: most agree RPT requirements should encourage peer-reviewed works of high quality, but in practice, the value of publications is often assessed using shortcuts such as the prestige of the publication venue, rather than on the quality and rigor of peer review of each individual item. Open access and online publishing have made these issues even murkier due to misconceptions about peer review practices and concerns about predatory online publishers, which leaves traditional publishing formats the most desired despite their restricted circulation. And, efforts to replace journal-level measures such as the impact factor with more precise article-level metrics (e.g., citation counts and altmetrics) have been slow to integrate with the RPT process. Questions remain as to whether, or how, RPT practices should be changed to better reflect faculty work patterns and reduce pressure to publish in only the most prestigious traditional formats. To determine the most useful way to change RPT, we need to assess further the needs and perceptions of faculty and administrators, and gain a better understanding of the level of influence of written RPT guidelines and policy in an often vague process that is meant to allow for flexibility in assessing individuals.
| S-EPMC6325612 | biostudies-other
Project description:OBJECTIVE:To determine the presence of a set of pre-specified traditional and non-traditional criteria used to assess scientists for promotion and tenure in faculties of biomedical sciences among universities worldwide. DESIGN:Cross sectional study. SETTING:International sample of universities. PARTICIPANTS:170 randomly selected universities from the Leiden ranking of world universities list. MAIN OUTCOME MEASURE:Presence of five traditional (for example, number of publications) and seven non-traditional (for example, data sharing) criteria in guidelines for assessing assistant professors, associate professors, and professors and the granting of tenure in institutions with biomedical faculties. RESULTS:A total of 146 institutions had faculties of biomedical sciences, and 92 had eligible guidelines available for review. Traditional criteria of peer reviewed publications, authorship order, journal impact factor, grant funding, and national or international reputation were mentioned in 95% (n=87), 37% (34), 28% (26), 67% (62), and 48% (44) of the guidelines, respectively. Conversely, among non-traditional criteria, only citations (any mention in 26%; n=24) and accommodations for employment leave (37%; 34) were relatively commonly mentioned. Mention of alternative metrics for sharing research (3%; n=3) and data sharing (1%; 1) was rare, and three criteria (publishing in open access mediums, registering research, and adhering to reporting guidelines) were not found in any guidelines reviewed. Among guidelines for assessing promotion to full professor, traditional criteria were more commonly reported than non-traditional criteria (traditional criteria 54.2%, non-traditional items 9.5%; mean difference 44.8%, 95% confidence interval 39.6% to 50.0%; P=0.001). Notable differences were observed across continents in whether guidelines were accessible (Australia 100% (6/6), North America 97% (28/29), Europe 50% (27/54), Asia 58% (29/50), South America 17% (1/6)), with more subtle differences in the use of specific criteria. CONCLUSIONS:This study shows that the evaluation of scientists emphasises traditional criteria as opposed to non-traditional criteria. This may reinforce research practices that are known to be problematic while insufficiently supporting the conduct of better quality research and open science. Institutions should consider incentivising non-traditional criteria. STUDY REGISTRATION:Open Science Framework (https://osf.io/26ucp/?view_only=b80d2bc7416543639f577c1b8f756e44).
| S-EPMC7315647 | biostudies-literature
Project description:BackgroundElectronic educational (e-learning) technology usage continues to grow. Many medical journals operate companion blogs (an application of e-learning technology) that enable rapid dissemination of scientific knowledge and discourse. Faculty members participating in promotion and tenure academic tracks spend valuable time and effort contributing, editing, and directing these medical journal blogs.ObjectiveWe sought to understand whether chairs of medicine and pediatric departments acknowledge blog authorship as academic achievement.MethodsThe authors surveyed 267 chairs of US and Canadian medicine and pediatric departments regarding their attitudes toward the role of faculty participation in e-learning and blogging in the promotion and tenure process. The survey completion rate was 22.8% (61/267).ResultsA majority of respondents (87%, 53/61) viewed educational scholarship as either important or very important for promotion. However, only 23% (14/61) perceived importance to faculty effort in producing content for journal-based blogs. If faculty were to participate in blog authorship, 72% (44/61) of surveyed chairs favored involvement in a journal-based versus a society-based or a personal (nonaffiliated) blog. We identified a "favorable group" of chairs (19/59, 32%), who rated leadership roles in e-learning tools as important or very important, and an "unfavorable group" of chairs (40/59, 68%), who rated leadership roles in e-learning tools as somewhat important or not important. The favorable group were more likely to be aware of faculty bloggers within their departments (58%, 11/19 vs 25%, 10/40), viewed serving on editorial boards of e-learning tools more favorably (79%, 15/19 vs 31%, 12/39), and were more likely to value effort spent contributing to journal-based blogs (53%, 10/19 vs 10%, 4/40).ConclusionsOur findings demonstrate that although the majority of department chairs value educational scholarship, only a minority perceive value in faculty blogging effort.
| S-EPMC5041355 | biostudies-literature
Project description:Assessment of researchers is necessary for decisions of hiring, promotion, and tenure. A burgeoning number of scientific leaders believe the current system of faculty incentives and rewards is misaligned with the needs of society and disconnected from the evidence about the causes of the reproducibility crisis and suboptimal quality of the scientific publication record. To address this issue, particularly for the clinical and life sciences, we convened a 22-member expert panel workshop in Washington, DC, in January 2017. Twenty-two academic leaders, funders, and scientists participated in the meeting. As background for the meeting, we completed a selective literature review of 22 key documents critiquing the current incentive system. From each document, we extracted how the authors perceived the problems of assessing science and scientists, the unintended consequences of maintaining the status quo for assessing scientists, and details of their proposed solutions. The resulting table was used as a seed for participant discussion. This resulted in six principles for assessing scientists and associated research and policy implications. We hope the content of this paper will serve as a basis for establishing best practices and redesigning the current approaches to assessing scientists by the many players involved in that process.
| S-EPMC5892914 | biostudies-literature
Project description:Much of the work done by faculty at both public and private universities has significant public dimensions: it is often paid for by public funds; it is often aimed at serving the public good; and it is often subject to public evaluation. To understand how the public dimensions of faculty work are valued, we analyzed review, promotion, and tenure documents from a representative sample of 129 universities in the US and Canada. Terms and concepts related to public and community are mentioned in a large portion of documents, but mostly in ways that relate to service, which is an undervalued aspect of academic careers. Moreover, the documents make significant mention of traditional research outputs and citation-based metrics: however, such outputs and metrics reward faculty work targeted to academics, and often disregard the public dimensions. Institutions that seek to embody their public mission could therefore work towards changing how faculty work is assessed and incentivized.
| S-EPMC6391063 | biostudies-literature
Project description:Analyses of the Journal Impact Factor (JIF) have grown to be a major topic in scientometric literature. Despite widespread and justified critique concerning the JIF and its application, the size of a journal as a predictor for its longitudinal variability-or stability-on a long-term level has not yet comprehensively been analyzed. This study aims to provide robust evidence for an association between JIF variability and the size of journals, expressed by the number of published articles (citable items). For this purpose, the complete set of journals included in the Incite Journal Citation Reports (JCR) with an JIF in the 2017 JCR edition (n = 8750) were analyzed for the association between journal size and longitudinal JIF dynamics. Our results, based on n = 4792 journals with a complete JIF data set over the timespan of 12 annual JIF changes show that larger journals publishing more citable items experience smaller annual changes of the JIF than smaller journals, yet with this association being reversed for journals with a very large number of total cites. Consequently and in accordance with the genuine intention of the JIF to serve as a basis for decisions on journal subscriptions, evaluation of current changes of the JIF have to be accompanied by consideration of the journal's size in order to be accurate and sensible.
| S-EPMC6874322 | biostudies-literature
Project description:BackgroundWithin increasingly constrained school timetables, interventions that integrate academic and health education to reduce substance use and violence may hold promise as a category of intervention that can positively affect both academic and health outcomes. There are no current systematic reviews exploring the effectiveness of such interventions or factors that affect their implementation.MethodsA total of 19 bibliographic databases and 32 websites were searched. References were also extracted from the reference lists of included studies, and experts and authors were contacted to identify relevant studies. We included reports with no restrictions on language or date. References were screened on title/abstract and those not thus excluded were screened on full report. Data extraction and appraisal followed the Critical Appraisal Skills Programme, Evidence for Policy and Practice Information and Co-ordinating Centre and Cochrane tools. Extracted process data were qualitatively meta-synthesised for common themes.ResultsSeventy-eight thousand four hundred fifty-one unique references were identified, and 62 reports were included. A total of 16 reports (reporting on 15 studies of 12 interventions) evaluated process. Key facilitators of integrated academic and health curricula were supportive senior management and alignment of the intervention with school ethos; a positive teaching environment, including positive perceptions around the ability to be flexible in the adaptation and delivery of integrated academic and health curricula; positive pre-existing student and teacher attitudes towards intervention content; and parental support of interventions, largely through reinforcement of messaging at home. Important barriers were over-burdened teachers, with little time to learn and implement integrated curricula.ConclusionSeveral useful facilitating and inhibiting factors linked to the implementation of interventions that integrate academic and health education for reduced substance use and/or violence were identified, providing tentative but insightful evidence of context-specific issues that may impact intervention success. However, overall, there is still a considerable gap in our understanding of how to achieve the successful implementation of these interventions.
| S-EPMC6284294 | biostudies-literature
Project description:To correct the incongruence of document types between the numerator and denominator in the traditional impact factor (IF), we make a corresponding adjustment to its formula and present five corrective IFs: IFTotal/Total, IFTotal/AREL, IFAR/AR, IFAREL/AR, and IFAREL/AREL. Based on a survey of researchers in the fields of ophthalmology and mathematics, we obtained the real impact ranking of sample journals in the minds of peer experts. The correlations between various IFs and questionnaire score were analyzed to verify their journal evaluation effects. The results show that it is scientific and reasonable to use five corrective IFs for journal evaluation for both ophthalmology and mathematics. For ophthalmology, the journal evaluation effects of the five corrective IFs are superior than those of traditional IF: the corrective effect of IFAR/AR is the best, IFAREL/AR is better than IFTotal/Total, followed by IFTotal/AREL, and IFAREL/AREL. For mathematics, the journal evaluation effect of traditional IF is superior than those of the five corrective IFs: the corrective effect of IFTotal/Total is best, IFAREL/AR is better than IFTotal/AREL and IFAREL/AREL, and the corrective effect of IFAR/AR is the worst. In conclusion, not all disciplinary journal IF need correction. The results in the current paper show that to correct the IF of ophthalmologic journals may be valuable, but it seems to be meaningless for mathematic journals.
| S-EPMC4792445 | biostudies-literature
Project description:Using an online survey of academics at 55 randomly selected institutions across the US and Canada, we explore priorities for publishing decisions and their perceived importance within review, promotion, and tenure (RPT). We find that respondents most value journal readership, while they believe their peers most value prestige and related metrics such as impact factor when submitting their work for publication. Respondents indicated that total number of publications, number of publications per year, and journal name recognition were the most valued factors in RPT. Older and tenured respondents (most likely to serve on RPT committees) were less likely to value journal prestige and metrics for publishing, while untenured respondents were more likely to value these factors. These results suggest disconnects between what academics value versus what they think their peers value, and between the importance of journal prestige and metrics for tenured versus untenured faculty in publishing and RPT perceptions.
| S-EPMC7065820 | biostudies-literature