Association of percentile ranking with citation impact and productivity in a large cohort of de novo NIMH-funded R01 grants.
Ontology highlight
ABSTRACT:
SUBMITTER: Doyle JM
PROVIDER: S-EPMC5526589 | biostudies-literature | 2015 Sep
REPOSITORIES: biostudies-literature
Publications
Association of percentile ranking with citation impact and productivity in a large cohort of de novo NIMH-funded R01 grants.
Doyle J M JM Quinn K K Bodenstein Y A YA Wu C O CO Danthi N N Lauer M S MS
Molecular psychiatry 20150602 9
Previous reports from National Institutes of Health and National Science Foundation have suggested that peer review scores of funded grants bear no association with grant citation impact and productivity. This lack of association, if true, may be particularly concerning during times of increasing competition for increasingly limited funds. We analyzed the citation impact and productivity for 1755 de novo investigator-initiated R01 grants funded for at least 2 years by National Institute of Menta ...[more]
Similar Datasets
Project description:RationaleFunding decisions for cardiovascular R01 grant applications at the National Heart, Lung, and Blood Institute (NHLBI) largely hinge on percentile rankings. It is not known whether this approach enables the highest impact science.ObjectiveOur aim was to conduct an observational analysis of percentile rankings and bibliometric outcomes for a contemporary set of funded NHLBI cardiovascular R01 grants.Methods and resultsWe identified 1492 investigator-initiated de novo R01 grant applications that were funded between 2001 and 2008 and followed their progress for linked publications and citations to those publications. Our coprimary end points were citations received per million dollars of funding, citations obtained <2 years of publication, and 2-year citations for each grant's maximally cited paper. In 7654 grant-years of funding that generated $3004 million of total National Institutes of Health awards, the portfolio yielded 16 793 publications that appeared between 2001 and 2012 (median per grant, 8; 25th and 75th percentiles, 4 and 14; range, 0-123), which received 2 224 255 citations (median per grant, 1048; 25th and 75th percentiles, 492 and 1932; range, 0-16 295). We found no association between percentile rankings and citation metrics; the absence of association persisted even after accounting for calendar time, grant duration, number of grants acknowledged per paper, number of authors per paper, early investigator status, human versus nonhuman focus, and institutional funding. An exploratory machine learning analysis suggested that grants with the best percentile rankings did yield more maximally cited papers.ConclusionsIn a large cohort of NHLBI-funded cardiovascular R01 grants, we were unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics.
| S-EPMC3959724 | biostudies-literature
Project description:RATIONALE:The American Recovery and Reinvestment Act (ARRA) allowed National Heart, Lung, and Blood Institute to fund R01 grants that fared less well on peer review than those funded by meeting a payline threshold. It is not clear whether the sudden availability of additional funding enabled research of similar or lesser citation impact than already funded work. OBJECTIVE:To compare the citation impact of ARRA-funded de novo National Heart, Lung, and Blood Institute R01 grants with concurrent de novo National Heart, Lung, and Blood Institute R01 grants funded by standard payline mechanisms. METHODS AND RESULTS:We identified de novo (type 1) R01 grants funded by National Heart, Lung, and Blood Institute in fiscal year 2009: these included 458 funded by meeting Institute's published payline and 165 funded only because of ARRA funding. Compared with payline grants, ARRA grants received fewer total funds (median values, $1.03 versus $1.87 million; P<0.001) for a shorter duration (median values including no-cost extensions, 3.0 versus 4.9 years; P<0.001). Through May 2014, the payline R01 grants generated 3895 publications, whereas the ARRA R01 grants generated 996. Using the InCites database from Thomson-Reuters, we calculated a normalized citation impact for each grant by weighting each article for the number of citations it received normalizing for subject, article type, and year of publication. The ARRA R01 grants had a similar normalized citation impact per $1 million spent as the payline grants (median values [interquartile range], 2.15 [0.73-4.68] versus 2.03 [0.75-4.10]; P=0.61). The similar impact of the ARRA grants persisted even after accounting for potential confounders. CONCLUSIONS:Despite shorter durations and lower budgets, ARRA R01 grants had comparable citation outcomes per $million spent to that of contemporaneously funded payline R01 grants.
| S-EPMC4387375 | biostudies-literature
Project description:BackgroundEvaluating the quality and reliability of a de novo assembly and of single contigs in particular is challenging since commonly a ground truth is not readily available and numerous factors may influence results. Currently available procedures provide assembly scores but lack a comparative quality ranking of contigs within an assembly.ResultsWe present SuRankCo, which relies on a machine learning approach to predict quality scores for contigs and to enable the ranking of contigs within an assembly. The result is a sorted contig set which allows selective contig usage in downstream analysis. Benchmarking on datasets with known ground truth shows promising sensitivity and specificity and favorable comparison to existing methodology.ConclusionsSuRankCo analyzes the reliability of de novo assemblies on the contig level and thereby allows quality control and ranking prior to further downstream and validation experiments.
| S-EPMC4520199 | biostudies-literature
Project description:BackgroundThe widespread adoption of smartphones provides researchers with expanded opportunities for developing, testing and implementing interventions. National Institutes of Health (NIH) funds competitive, investigator-initiated grant applications. Funded grants represent the state of the science and therefore are expected to anticipate the progression of research in the near future.ObjectiveThe objective of this paper is to provide an analysis of the kinds of smartphone-based intervention apps funded in NIH research grants during the five-year period between 2014 and 2018.MethodsWe queried NIH Reporter to identify candidate funded grants that addressed mHealth and the use of smartphones. From 1524 potential grants, we identified 397 that met the requisites of including an intervention app. Each grant's abstract was analyzed to understand the focus of intervention. The year of funding, type of activity (eg, R01, R34, and so on) and funding were noted.ResultsWe identified 13 categories of strategies employed in funded smartphone intervention apps. Most grants included either one (35.0%) or two (39.0%) intervention approaches. These included artificial intelligence (57 apps), bionic adaptation (33 apps), cognitive and behavioral therapies (68 apps), contingency management (24 apps), education and information (85 apps), enhanced motivation (50 apps), facilitating, reminding and referring (60 apps), gaming and gamification (52 apps), mindfulness training (18 apps), monitoring and feedback (192 apps), norm setting (7 apps), skills training (85 apps) and social support and social networking (59 apps). The most frequently observed grant types included Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) grants (40.8%) and Research Project Grants (R01s) (26.2%). The number of grants funded increased through the five-year period from 60 in 2014 to 112 in 2018.ConclusionsSmartphone intervention apps are increasingly competitive for NIH funding. They reflect a wide diversity of approaches that have significant potential for use in applied settings.
| S-EPMC6690163 | biostudies-literature
Project description:BackgroundHealth economics research is an integral part of the transdisciplinary research supported by the National Cancer Institute (NCI). To better understand NCI activities in this area, we conducted a portfolio analysis of funded NCI grants including health economics research.MethodsWe examined all competitive grants funded by NCI from fiscal years 2015 to 2020 that included economic analyses or outcomes. Grant titles, abstracts, and specific aims were independently reviewed by 2 study team members; content of included grants was then coded for analysis.ResultsA total 212 grants were identified from searches; 146 of these included economic analyses and were included in the portfolio analysis. These 146 grants represent approximately 0.9% of all NCI competitively funded grants awarded 2015-2020. Of these grants, 100 were R01 awards, representing approximately 2.4% of all NCI R01 grants funded 2015-2020. The most common study type was interventional randomized controlled-trial, followed by simulation or model. Screening and prevention were the most frequent grant cancer continuum topic; survivorship was included in only 16 grants (11.0%). Cost-effectiveness analysis was the most frequently listed economic outcome (97 grants, 66.4%), whereas policy impact (20 grants, 13.7%) and financial hardship (15 grants, 10.3%) were less-frequently included economic outcomes. However, economic outcomes differed by cancer control continuum topic, with financial hardship being included in a greater proportion of treatment and survivorship grants.ConclusionsAlthough relatively small, the NCI portfolio of funded grants including economic analyses is diverse, covering a range of cancer types, methods, and economic outcomes, and increasing over time.
| S-EPMC9255945 | biostudies-literature
Project description:ImportanceScientific publication is an important tool for knowledge dissemination and career advancement, but authors affiliated with institutions in low- and middle-income countries (LMICs) are historically underrepresented on publications.ObjectiveTo assess the country income level distribution of author affiliations for publications resulting from National Cancer Institute (NCI)-supported extramural grants between 2015 and 2019, with international collaborating institutions exclusively in 1 or more LMICs.Design and settingThis cross-sectional study assessed authorship on publications resulting from NCI-funded grants between October 1, 2015, and September 30, 2019. Grants with collaborators in LMICs were identified in the National Institutes of Health (NIH) Query/View/Report and linked to publications using Dimensions for NIH, published between 2011 and 2020. Statistical analysis was performed from May 2021 to July 2022.Main outcomes and measuresAuthor institutional affiliation was used to classify author country and related income level as defined by the World Bank. Relative citation ratio and Altmetric data from Dimensions for NIH were used to compare citation impact measures using the Wilcoxon rank sum test.ResultsIn this cross-sectional study, 159 grants were awarded to US institutions with collaborators in LMICs, and 5 grants were awarded directly to foreign institutions. These 164 grants resulted in 2428 publications, of which 1242 (51%) did not include any authors affiliated with an institution in an LMIC. In addition, 1884 (78%) and 2009 (83%) publications had a first or last author, respectively, affiliated with a high-income country (HIC). Publications with HIC-affiliated last authors also demonstrated greater citation impact compared with publications with LMIC-affiliated last authors as measured by relative citation ratios and Altmetric Attention Scores; publications with HIC-affiliated first authors also had higher Altmetric Attention Scores.Conclusions and relevanceThis cross-sectional study suggests that LMIC-affiliated authors were underrepresented on publications resulting from NCI-funded grants involving LMICs. It is critical to promote equitable scientific participation by LMIC institutions in cancer research, including through current and planned programs led by the NCI.
| S-EPMC10980966 | biostudies-literature
Project description:Federal investment in survivorship science has grown markedly since the National Cancer Institute's creation of the Office of Cancer Survivorship in 1996. To describe the nature of this research, provide a benchmark, and map new directions for the future, a portfolio analysis of National Institutes of Health-wide survivorship grants was undertaken for fiscal year 2016. Applying survivorship-relevant terms, a search was conducted using the National Institutes of Health Information for Management, Planning, Analysis and Coordination grants database. Grants identified were reviewed for inclusion and categorized by grant mechanism used, funding agency, and principal investigator characteristics. Trained pairs of coders classified each grant by focus and design (observational vs interventional), population studied, and outcomes examined. A total of 215 survivorship grants were identified; 7 were excluded for lack of fit and 2 for nonresearch focus. Forty-one (19.7%) representing training grants (n = 38) or conference grants (n = 3) were not coded. Of the remaining 165 grants, most (88.5%) were funded by the National Cancer Institute; used the large, investigator-initiated (R01) mechanism (66.7%); focused on adult survivors alone (84.2%), often breast cancer survivors (47.3%); were observational in nature (57.3%); and addressed a broad array of topics, including psychosocial and physiologic outcomes, health behaviors, patterns of care, and economic/employment outcomes. Grants were led by investigators from diverse backgrounds, 28.4% of whom were early in their career. Present funding patterns, many stable since 2006, point to the need to expand research to include different cancer sites, greater ethnoculturally diverse samples, and older (>65 years) as well as longer-term (>5 years) survivors and address effects of newer therapies.
| S-EPMC6657281 | biostudies-literature
Project description:Aging affects most living organisms and includes the processes that reduce health and survival. The chronological and the biological age of individuals can differ remarkably, and there is a lack of reliable biomarkers to monitor the consequences of aging. In this review we give an overview of commonly mentioned and frequently used potential aging-related biomarkers. We were interested in biomarkers of aging in general and in biomarkers related to cellular senescence in particular. To answer the question whether a biological feature is relevant as a potential biomarker of aging or senescence in the scientific community we used the PICO strategy known from evidence-based medicine. We introduced two scoring systems, aimed at reflecting biomarker relevance and measurement effort, which can be used to support study designs in both clinical and research settings.
| S-EPMC8176216 | biostudies-literature
Project description:BackgroundTelehealth use increased during the COVID-19 pandemic and remains a complementary source of cancer care delivery. Understanding research funding trends in cancer-related telehealth can highlight developments in this area of science and identify future opportunities.MethodsApplications funded by the US National Cancer Institute (NCI) between fiscal years 2016 and 2022 and focused on synchronous patient-provider telehealth were analyzed for grant characteristics (eg, funding mechanism), cancer focus (eg, cancer type), and study features (eg, type of telehealth service). Of 106 grants identified initially, 60 were retained for coding after applying exclusion criteria.ResultsAlmost three-quarters (73%) of telehealth grants were funded during fiscal years 2020-2022. Approximately 67% were funded through R01 or R37 mechanism and implemented as randomized controlled trials (63%). Overall, telehealth grants commonly focused on treatment (30%) and survivorship (43%); breast cancer (12%), hematologic malignancies (10%), and multiple cancer sites (27%); and health disparity populations (ie, minorities, rural residents) (73%). Both audio and video telehealth were common (65%), as well as accompanying mHealth apps (20%). Telehealth services centered on psychosocial care, self-management, and supportive care (88%); interventions were commonly delivered by mental health professionals (30%).ConclusionNCI has observed an increase in funded synchronous patient-provider telehealth grants. Trends indicate an evolution of awards that have expanded across the cancer control continuum, applied rigorous study designs, incorporated additional digital technologies, and focused on populations recognized for disparate cancer outcomes. As telehealth is integrated into routine cancer care delivery, additional research evidence will be needed to inform clinical practice.
| S-EPMC11207831 | biostudies-literature
Project description:PurposeImplementation science offers methods to evaluate the translation of genomic medicine research into practice. The extent to which the National Institutes of Health (NIH) human genomics grant portfolio includes implementation science is unknown. This brief report's objective is to describe recently funded implementation science studies in genomic medicine in the NIH grant portfolio, and identify remaining gaps.MethodsWe identified investigator-initiated NIH research grants on implementation science in genomic medicine (funding initiated 2012-2016). A codebook was adapted from the literature, three authors coded grants, and descriptive statistics were calculated for each code.ResultsForty-two grants fit the inclusion criteria (~1.75% of investigator-initiated genomics grants). The majority of included grants proposed qualitative and/or quantitative methods with cross-sectional study designs, and described clinical settings and primarily white, non-Hispanic study populations. Most grants were in oncology and examined genetic testing for risk assessment. Finally, grants lacked the use of implementation science frameworks, and most examined uptake of genomic medicine and/or assessed patient-centeredness.ConclusionWe identified large gaps in implementation science studies in genomic medicine in the funded NIH portfolio over the past 5 years. To move the genomics field forward, investigator-initiated research grants should employ rigorous implementation science methods within diverse settings and populations.
| S-EPMC5920776 | biostudies-literature