Dataset Information

Scholarly Productivity Evaluation of KL2 Scholars Using Bibliometrics and Federal Follow-on Funding: Cross-Institution Study.

ABSTRACT:

Background

Evaluating outcomes of the clinical and translational research (CTR) training of a Clinical and Translational Science Award (CTSA) hub (eg, the KL2 program) requires the selection of reliable, accessible, and standardized measures. As measures of scholarly success usually focus on publication output and extramural funding, CTSA hubs have started to use bibliometrics to evaluate the impact of their supported scholarly activities. However, the evaluation of KL2 programs across CTSAs is limited, and the use of bibliometrics and follow-on funding is minimal.

Objective

This study seeks to evaluate scholarly productivity, impact, and collaboration using bibliometrics and federal follow-on funding of KL2 scholars from 3 CTSA hubs and to define and assess CTR training success indicators.

Methods

The sample included KL2 scholars from 3 CTSA institutions (A-C). Bibliometric data for each scholar in the sample were collected from both SciVal and iCite, including scholarly productivity, citation impact, and research collaboration. Three federal follow-on funding measures (at the 5-year, 8-year, and overall time points) were collected internally and confirmed by examining a federal funding database. Both descriptive and inferential statistical analyses were computed using SPSS to assess the bibliometric and federal follow-on funding results.

Results

A total of 143 KL2 scholars were included in the sample with relatively equal groups across the 3 CTSA institutions. The included KL2 scholars produced more publications and citation counts per year on average at the 8-year time point (3.75 publications and 26.44 citation counts) than the 5-year time point (3.4 publications vs 26.16 citation counts). Overall, the KL2 publications from all 3 institutions were cited twice as much as others in their fields based on the relative citation ratio. KL2 scholars published work with researchers from other US institutions over 2 times (5-year time point) or 3.5 times (8-year time point) more than others in their research fields. Within 5 years and 8 years postmatriculation, 44.1% (63/143) and 51.7% (74/143) of KL2 scholars achieved federal funding, respectively. The KL2-scholars of Institution C had a significantly higher citation rate per publication than the other institutions (P<.001). Institution A had a significantly lower rate of nationally field-weighted collaboration than did the other institutions (P<.001). Institution B scholars were more likely to have received federal funding than scholars at Institution A or C (P<.001).

Conclusions

Multi-institutional data showed a high level of scholarly productivity, impact, collaboration, and federal follow-on funding achieved by KL2 scholars. This study provides insights on the use of bibliometric and federal follow-on funding data to evaluate CTR training success across institutions. CTSA KL2 programs and other CTR career training programs can benefit from these findings in terms of understanding metrics of career success and using that knowledge to develop highly targeted strategies to support early-stage career development of CTR investigators.

SUBMITTER: Qua K

PROVIDER: S-EPMC8515229 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ImportanceThe US leads the world in the raw number of incarcerated persons as well as the rate of incarceration, with detrimental effects on individual-, family-, community-, and population-level health; as such, federal research has a critical role in documenting and addressing the health-related impacts of the US criminal legal system. How often incarceration-related research is funded at the National Institutes of Health (NIH), National Science Foundation (NSF), and US Department of Justice (DOJ) levels has a direct association with the public attention given to mass incarceration as well as the efficacy of strategies to mitigate negative effects and poor health related to incarceration.ObjectiveTo understand how many incarceration-related projects have been funded at the NIH, NSF, and DOJ.Design, setting, and participantsThis cross-sectional study used public historical project archives to search for relevant incarceration-related keywords (eg, incarceration, prison, parole) since January 1, 1985 (NIH and NSF), and since January 1, 2008 (DOJ). Quotations and Boolean operator logic were used. All searches and counts were conducted and double verified by 2 coauthors between December 12 and 17, 2022.Main outcomes and measuresNumber and prevalence of funded projects related to incarceration and prison keywords.ResultsThe term incarceration resulted in 3540 of 3 234 159 total project awards (0.11%) and prisoner-related terms resulted in 11 455 total project awards (0.35%) across the 3 federal agencies since 1985. Nearly a tenth of all projects funded at NIH since 1985 related to education (256 584 [9.62%]) compared with only 3373 projects (0.13%) that related to criminal legal or criminal justice or correctional system and 18 projects (0.0007%) that related to incarcerated parents. Only 1857 (0.07%) of all NIH-funded projects have been funded related to racism since 1985.Conclusions and relevanceThis cross-sectional study found that a very low number of projects about incarceration have historically been funded at the NIH, DOJ, and NSF. These findings reflect a dearth of federally funded studies investigating the effects of mass incarceration or intervention strategies to mitigate adverse effects. Given the consequences of the criminal legal system, it is undoubtedly time for researchers, and our nation, to invest more resources into studying whether this system should be maintained, the intergenerational effects of mass incarceration, and strategies to best mitigate its impact on public health.

Project description:ImportanceBoth citation and funding metrics converge in shaping current perceptions of academic success.ObjectiveTo evaluate what proportion of the most-cited US-based scientists are funded by biomedical federal agencies and whether funded scientists are more cited than nonfunded ones.Design, setting, and participantsThis survey study used linkage of a Scopus-based database on top-cited US researchers (according to a composite citation metric) and the National Institutes of Health RePORTER database of federal funding (33 biomedical federal agencies). Matching was based on name and institution. US-based top-cited scientists who were allocated to any of 69 scientific subfields highly related to biomedicine were considered in the main analysis. Data were downloaded on June 11, 2022.Main outcomes and measuresProportion of US-based top-cited biomedical scientists who had any (1996-2022), recent (2015-2022), and current (2021-2022) funding. Comparisons of funded and nonfunded scientists assessed total citations and a composite citation index.ResultsThere were 204 603 records in RePORTER (1996-2022) and 75 316 US-based top-cited scientists in the career-long citation database; 40 887 scientists were included in the main analysis. The proportion of US-based top-cited biomedical scientists (according to career-long citation impact) who had received any federal funding from biomedical research agencies was 62.7% (25 650 of 40 887) for any funding (1996-2022), 23.1% (9427 of 40 887) for recent funding (2015-2022), and 14.1% (5778 of 40 887) for current funding (2021-2022). Respective proportions were 64.8%, 31.4%, and 20.9%, for top-cited scientists according to recent single-year citation impact. There was large variability across scientific subfields (eg, current funding: 31% of career-long impact top-cited scientists in geriatrics, 30% in bioinformatics and 29% in developmental biology, but 0% in legal and forensic medicine, general psychology and cognitive sciences, and gender studies). Funded top-cited researchers were overall more cited than nonfunded top-cited scientists (median [IQR], 9594 [5650-1703] vs 5352 [3057-9890] citations; P < .001) and substantial difference remained after adjusting for subfield and years since first publication. Differences were more prominent in some specific biomedical subfields.Conclusions and relevanceIn this survey study, biomedical federal funding had offered support to approximately two-thirds of the top-cited biomedical scientists at some point during the last quarter century, but only a small minority of top-cited scientists had current federal biomedical funding. The large unevenness across subfields needs to be addressed with ways that improve equity, efficiency, excellence, and translational potential.

Project description:IntroductionThe emergence of popular online rating websites, social media platforms, and public databases for industry payments and scholarly outputs provide a complete physician online presence which may guide choice and satisfaction.MethodsWebsites of all U.S. otolaryngology academic institutions were queried for fellowship-trained rhinologists. Additional well-known and academically active rhinologists were identified by the senior author. Online ratings and comments were collected from Google, Healthgrades, Vitals, and RateMD websites, and weighted rating scores (RS) were calculated on a 1-5 scale.ResultsA total of 210 rhinologists with 16 ± 9 years of practice were included, where 6901 online ratings (33 ± 47 per rhinologist) provided an average RS of 4.3 ± 0.6. RS was not different according to gender (p = 0.58), geographic quartile (p = 0.48), social media presence (p = 0.41), or attending top-ranked medical school (p = 0.86) or residency programs (p = 0.89). Years of practice negatively correlated with RS (R = -0.22, p<0.01), and academic ranking significantly influenced RS, with professors, associate professors, and assistant professors scoring 4.1 ± 0.6, 4.3 ± 0.4, and 4.4 ± 0.6, respectively (p = 0.03). Of the 3,304 narrative comments analyzed (3.1 ± 11.6 per rhinologist), 76% (positive) and 7% (negative) had elements of clinical knowledge/outcomes, 56% (positive) and 7% (negative) of communication/bedside manner, and 9% (positive) and 7% (negative) of office staff, cost, and wait-time. All negative comment categories had moderate negative correlation with RS, while positive comment categories regarding knowledge/competence and bedside manner weakly correlated with higher RS. Number of publications (48 ± 54) positively correlated with 2018 industry payments ($11,384 ± $19,025) among those receiving industry compensation >$300 (n = 113). Attending a top-ranked medical school was associated with higher industry payments (p<0.01) and H-index (p = 0.02).ConclusionAcademic rhinologists' online RS was not associated with gender, geographic location, or attending a top-ranked training program, and their scholarly productivity was significantly correlated with total industry payments.