Dataset Information

Scholarly Productivity Evaluation of KL2 Scholars Using Bibliometrics and Federal Follow-on Funding: Cross-Institution Study.

ABSTRACT:

Background

Evaluating outcomes of the clinical and translational research (CTR) training of a Clinical and Translational Science Award (CTSA) hub (eg, the KL2 program) requires the selection of reliable, accessible, and standardized measures. As measures of scholarly success usually focus on publication output and extramural funding, CTSA hubs have started to use bibliometrics to evaluate the impact of their supported scholarly activities. However, the evaluation of KL2 programs across CTSAs is limited, and the use of bibliometrics and follow-on funding is minimal.

Objective

This study seeks to evaluate scholarly productivity, impact, and collaboration using bibliometrics and federal follow-on funding of KL2 scholars from 3 CTSA hubs and to define and assess CTR training success indicators.

Methods

The sample included KL2 scholars from 3 CTSA institutions (A-C). Bibliometric data for each scholar in the sample were collected from both SciVal and iCite, including scholarly productivity, citation impact, and research collaboration. Three federal follow-on funding measures (at the 5-year, 8-year, and overall time points) were collected internally and confirmed by examining a federal funding database. Both descriptive and inferential statistical analyses were computed using SPSS to assess the bibliometric and federal follow-on funding results.

Results

A total of 143 KL2 scholars were included in the sample with relatively equal groups across the 3 CTSA institutions. The included KL2 scholars produced more publications and citation counts per year on average at the 8-year time point (3.75 publications and 26.44 citation counts) than the 5-year time point (3.4 publications vs 26.16 citation counts). Overall, the KL2 publications from all 3 institutions were cited twice as much as others in their fields based on the relative citation ratio. KL2 scholars published work with researchers from other US institutions over 2 times (5-year time point) or 3.5 times (8-year time point) more than others in their research fields. Within 5 years and 8 years postmatriculation, 44.1% (63/143) and 51.7% (74/143) of KL2 scholars achieved federal funding, respectively. The KL2-scholars of Institution C had a significantly higher citation rate per publication than the other institutions (P<.001). Institution A had a significantly lower rate of nationally field-weighted collaboration than did the other institutions (P<.001). Institution B scholars were more likely to have received federal funding than scholars at Institution A or C (P<.001).

Conclusions

Multi-institutional data showed a high level of scholarly productivity, impact, collaboration, and federal follow-on funding achieved by KL2 scholars. This study provides insights on the use of bibliometric and federal follow-on funding data to evaluate CTR training success across institutions. CTSA KL2 programs and other CTR career training programs can benefit from these findings in terms of understanding metrics of career success and using that knowledge to develop highly targeted strategies to support early-stage career development of CTR investigators.

SUBMITTER: Qua K

PROVIDER: S-EPMC8515229 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionThe emergence of popular online rating websites, social media platforms, and public databases for industry payments and scholarly outputs provide a complete physician online presence which may guide choice and satisfaction.MethodsWebsites of all U.S. otolaryngology academic institutions were queried for fellowship-trained rhinologists. Additional well-known and academically active rhinologists were identified by the senior author. Online ratings and comments were collected from Google, Healthgrades, Vitals, and RateMD websites, and weighted rating scores (RS) were calculated on a 1-5 scale.ResultsA total of 210 rhinologists with 16 ± 9 years of practice were included, where 6901 online ratings (33 ± 47 per rhinologist) provided an average RS of 4.3 ± 0.6. RS was not different according to gender (p = 0.58), geographic quartile (p = 0.48), social media presence (p = 0.41), or attending top-ranked medical school (p = 0.86) or residency programs (p = 0.89). Years of practice negatively correlated with RS (R = -0.22, p<0.01), and academic ranking significantly influenced RS, with professors, associate professors, and assistant professors scoring 4.1 ± 0.6, 4.3 ± 0.4, and 4.4 ± 0.6, respectively (p = 0.03). Of the 3,304 narrative comments analyzed (3.1 ± 11.6 per rhinologist), 76% (positive) and 7% (negative) had elements of clinical knowledge/outcomes, 56% (positive) and 7% (negative) of communication/bedside manner, and 9% (positive) and 7% (negative) of office staff, cost, and wait-time. All negative comment categories had moderate negative correlation with RS, while positive comment categories regarding knowledge/competence and bedside manner weakly correlated with higher RS. Number of publications (48 ± 54) positively correlated with 2018 industry payments ($11,384 ± $19,025) among those receiving industry compensation >$300 (n = 113). Attending a top-ranked medical school was associated with higher industry payments (p<0.01) and H-index (p = 0.02).ConclusionAcademic rhinologists' online RS was not associated with gender, geographic location, or attending a top-ranked training program, and their scholarly productivity was significantly correlated with total industry payments.

Project description:ObjectivesTo describe and compare the characteristics of scholars who reviewed for predatory or legitimate journals in terms of their sociodemographic characteristics and reviewing and publishing behaviour.DesignLinkage of random samples of predatory journals and legitimate journals of the Cabells Scholarly Analytics' journal lists with the Publons database, employing the Jaro-Winkler string metric. Descriptive analysis of sociodemographic characteristics and reviewing and publishing behaviour of scholars for whom reviews were found in the Publons database.SettingPeer review of journal articles.ParticipantsReviewers who submitted peer review reports to Publons.MeasurementsNumbers of reviews for predatory journals and legitimate journals per reviewer. Academic age of reviewers, the total number of reviews, number of publications and number of reviews and publications per year.ResultsAnalyses included 183 743 unique reviews submitted to Publons by 19 598 reviewers. Six thousand and seventy-seven reviews were for 1160 predatory journals (3.31% of all reviews) and 177 666 reviews for 6403 legitimate journals (96.69%). Most scholars never submitted reviews for predatory journals (90.0% of all scholars); few scholars (7.6%) reviewed occasionally or rarely (1.9%) for predatory journals. Very few scholars submitted reviews predominantly or exclusively for predatory journals (0.26% and 0.35%, respectively). The latter groups of scholars were of younger academic age and had fewer publications and reviews than the first groups. Regions with the highest shares of predatory reviews were sub-Saharan Africa (21.8% reviews for predatory journals), Middle East and North Africa (13.9%) and South Asia (7.0%), followed by North America (2.1%), Latin America and the Caribbean (2.1%), Europe and Central Asia (1.9%) and East Asia and the Pacific (1.5%).ConclusionTo tackle predatory journals, universities, funders and publishers need to consider the entire research workflow and educate reviewers on concepts of quality and legitimacy in scholarly publishing.