Dataset Information

Are university rankings useful to improve research? A systematic review.

ABSTRACT: Concerns about reproducibility and impact of research urge improvement initiatives. Current university ranking systems evaluate and compare universities on measures of academic and research performance. Although often useful for marketing purposes, the value of ranking systems when examining quality and outcomes is unclear. The purpose of this study was to evaluate usefulness of ranking systems and identify opportunities to support research quality and performance improvement.A systematic review of university ranking systems was conducted to investigate research performance and academic quality measures. Eligibility requirements included: inclusion of at least 100 doctoral granting institutions, be currently produced on an ongoing basis and include both global and US universities, publish rank calculation methodology in English and independently calculate ranks. Ranking systems must also include some measures of research outcomes. Indicators were abstracted and contrasted with basic quality improvement requirements. Exploration of aggregation methods, validity of research and academic quality indicators, and suitability for quality improvement within ranking systems were also conducted.A total of 24 ranking systems were identified and 13 eligible ranking systems were evaluated. Six of the 13 rankings are 100% focused on research performance. For those reporting weighting, 76% of the total ranks are attributed to research indicators, with 24% attributed to academic or teaching quality. Seven systems rely on reputation surveys and/or faculty and alumni awards. Rankings influence academic choice yet research performance measures are the most weighted indicators. There are no generally accepted academic quality indicators in ranking systems.No single ranking system provides a comprehensive evaluation of research and academic quality. Utilizing a combined approach of the Leiden, Thomson Reuters Most Innovative Universities, and the SCImago ranking systems may provide institutions with a more effective feedback for research improvement. Rankings which extensively rely on subjective reputation and "luxury" indicators, such as award winning faculty or alumni who are high ranking executives, are not well suited for academic or research performance improvement initiatives. Future efforts should better explore measurement of the university research performance through comprehensive and standardized indicators. This paper could serve as a general literature citation when one or more of university ranking systems are used in efforts to improve academic prominence and research performance.

SUBMITTER: Vernon MM

PROVIDER: S-EPMC5841788 | biostudies-other | 2018

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Are university rankings useful to improve research? A systematic review.

Vernon Marlo M MM Balas E Andrew EA Momani Shaher S

PloS one 20180307 3

<h4>Introduction</h4>Concerns about reproducibility and impact of research urge improvement initiatives. Current university ranking systems evaluate and compare universities on measures of academic and research performance. Although often useful for marketing purposes, the value of ranking systems when examining quality and outcomes is unclear. The purpose of this study was to evaluate usefulness of ranking systems and identify opportunities to support research quality and performance improvemen ...[more]

PMID: 29513762

Similar Datasets

Project description:Carbon ion radiotherapy holds great promise for cancer therapy. Clinical data show that carbon ion radiotherapy is an effective treatment for tumors that are resistant to X-ray radiotherapy. Since 1994 in Japan, the National Institute of Radiological Sciences has been heading the development of carbon ion radiotherapy using the Heavy Ion Medical Accelerator in Chiba. The Gunma University Heavy Ion Medical Center (GHMC) was established in the year 2006 as a proof-of-principle institute for carbon ion radiotherapy with a view to facilitating the worldwide spread of compact accelerator systems. Along with the management of more than 1900 cancer patients to date, GHMC engages in translational research to improve the treatment efficacy of carbon ion radiotherapy. Research aimed at guiding patient selection is of utmost importance for making the most of carbon ion radiotherapy, which is an extremely limited medical resource. Intratumoral oxygen levels, radiation-induced cellular apoptosis, the capacity to repair DNA double-strand breaks, and the mutational status of tumor protein p53 and epidermal growth factor receptor genes are all associated with X-ray sensitivity. Assays for these factors are useful in the identification of X-ray-resistant tumors for which carbon ion radiotherapy would be beneficial. Research aimed at optimizing treatments based on carbon ion radiotherapy is also important. This includes assessment of dose fractionation, normal tissue toxicity, tumor cell motility, and bystander effects. Furthermore, the efficacy of carbon ion radiotherapy will likely be enhanced by research into combined treatment with other modalities such as chemotherapy. Several clinically available chemotherapeutic drugs (carboplatin, paclitaxel, and etoposide) and drugs at the developmental stage (Wee-1 and heat shock protein 90 inhibitors) show a sensitizing effect on tumor cells treated with carbon ions. Additionally, the efficacy of carbon ion radiotherapy can be improved by combining it with cancer immunotherapy. Clinical validation of preclinical findings is necessary to further improve the treatment efficacy of carbon ion radiotherapy.

Project description:BACKGROUND: A compelling ethical rationale supports patient engagement in healthcare research. It is also assumed that patient engagement will lead to research findings that are more pertinent to patients' concerns and dilemmas. However; it is unclear how to best conduct this process. In this systematic review we aimed to answer 4 key questions: what are the best ways to identify patient representatives? How to engage them in designing and conducting research? What are the observed benefits of patient engagement? What are the harms and barriers of patient engagement? METHODS: We searched MEDLINE, EMBASE, PsycInfo, Cochrane, EBSCO, CINAHL, SCOPUS, Web of Science, Business Search Premier, Academic Search Premier and Google Scholar. Included studies were published in English, of any size or design that described engaging patients or their surrogates in research design. We conducted an environmental scan of the grey literature and consulted with experts and patients. Data were analyzed using a non-quantitative, meta-narrative approach. RESULTS: We included 142 studies that described a spectrum of engagement. In general, engagement was feasible in most settings and most commonly done in the beginning of research (agenda setting and protocol development) and less commonly during the execution and translation of research. We found no comparative analytic studies to recommend a particular method. Patient engagement increased study enrollment rates and aided researchers in securing funding, designing study protocols and choosing relevant outcomes. The most commonly cited challenges were related to logistics (extra time and funding needed for engagement) and to an overarching worry of a tokenistic engagement. CONCLUSIONS: Patient engagement in healthcare research is likely feasible in many settings. However, this engagement comes at a cost and can become tokenistic. Research dedicated to identifying the best methods to achieve engagement is lacking and clearly needed.

Project description:BackgroundCultural competency training has been proposed as a way to improve patient outcomes. There is a need for evidence showing that these interventions reduce health disparities.ObjectiveThe objective was to conduct a systematic review addressing the effects of cultural competency training on patient-centered outcomes; assess quality of studies and strength of effect; and propose a framework for future research.DesignThe authors performed electronic searches in the MEDLINE/PubMed, ERIC, PsycINFO, CINAHL and Web of Science databases for original articles published in English between 1990 and 2010, and a bibliographic hand search. Studies that reported cultural competence educational interventions for health professionals and measured impact on patients and/or health care utilization as primary or secondary outcomes were included.MeasurementsFour authors independently rated studies for quality using validated criteria and assessed the training effect on patient outcomes. Due to study heterogeneity, data were not pooled; instead, qualitative synthesis and analysis were conducted.ResultsSeven studies met inclusion criteria. Three involved physicians, two involved mental health professionals and two involved multiple health professionals and students. Two were quasi-randomized, two were cluster randomized, and three were pre/post field studies. Study quality was low to moderate with none of high quality; most studies did not adequately control for potentially confounding variables. Effect size ranged from no effect to moderately beneficial (unable to assess in two studies). Three studies reported positive (beneficial) effects; none demonstrated a negative (harmful) effect.ConclusionThere is limited research showing a positive relationship between cultural competency training and improved patient outcomes, but there remains a paucity of high quality research. Future work should address challenges limiting quality. We propose an algorithm to guide educators in designing and evaluating curricula, to rigorously demonstrate the impact on patient outcomes and health disparities.

Project description:BACKGROUND: Depression and anxiety are common mental health difficulties experienced by university students and can impair academic and social functioning. Students are limited in seeking help from professionals. As university students are highly connected to digital technologies, Web-based and computer-delivered interventions could be used to improve students' mental health. The effectiveness of these intervention types requires investigation to identify whether these are viable prevention strategies for university students. OBJECTIVE: The intent of the study was to systematically review and analyze trials of Web-based and computer-delivered interventions to improve depression, anxiety, psychological distress, and stress in university students. METHODS: Several databases were searched using keywords relating to higher education students, mental health, and eHealth interventions. The eligibility criteria for studies included in the review were: (1) the study aimed to improve symptoms relating to depression, anxiety, psychological distress, and stress, (2) the study involved computer-delivered or Web-based interventions accessed via computer, laptop, or tablet, (3) the study was a randomized controlled trial, and (4) the study was trialed on higher education students. Trials were reviewed and outcome data analyzed through random effects meta-analyses for each outcome and each type of trial arm comparison. Cochrane Collaboration risk of bias tool was used to assess study quality. RESULTS: A total of 17 trials were identified, in which seven were the same three interventions on separate samples; 14 reported sufficient information for meta-analysis. The majority (n=13) were website-delivered and nine interventions were based on cognitive behavioral therapy (CBT). A total of 1795 participants were randomized and 1480 analyzed. Risk of bias was considered moderate, as many publications did not sufficiently report their methods and seven explicitly conducted completers' analyses. In comparison to the inactive control, sensitivity meta-analyses supported intervention in improving anxiety (pooled standardized mean difference [SMD] -0.56; 95% CI -0.77 to -0.35, P<.001), depression (pooled SMD -0.43; 95% CI -0.63 to -0.22, P<.001), and stress (pooled SMD -0.73; 95% CI -1.27 to -0.19, P=.008). In comparison to active controls, sensitivity analyses did not support either condition for anxiety (pooled SMD -0.18; 95% CI -0.98 to 0.62, P=.66) or depression (pooled SMD -0.28; 95% CI -0.75 to -0.20, P=.25). In contrast to a comparison intervention, neither condition was supported in sensitivity analyses for anxiety (pooled SMD -0.10; 95% CI -0.39 to 0.18, P=.48) or depression (pooled SMD -0.33; 95% CI -0.43 to 1.09, P=.40). CONCLUSIONS: The findings suggest Web-based and computer-delivered interventions can be effective in improving students' depression, anxiety, and stress outcomes when compared to inactive controls, but some caution is needed when compared to other trial arms and methodological issues were noticeable. Interventions need to be trialed on more heterogeneous student samples and would benefit from user evaluation. Future trials should address methodological considerations to improve reporting of trial quality and address post-intervention skewed data.

Project description:Depression is a major public health problem among youth, currently estimated to affect as many as 9% of US children and adolescents. The recognition that both genes (nature) and environments (nurture) are important for understanding the etiology of depression has led to a rapid growth in research exploring gene-environment interactions (GxE). However, there has been no systematic review of GxE in youth depression to date.The goal of this article was to systematically review evidence on the contribution of GxE to the risk of child and adolescent depression. Through a search of PubMed and PsycINFO databases to 1 April 2010, we identified 20 candidate gene-environment interaction studies focused on depression in youth (up to age 26) and compared each study in terms of the following characteristics: research design and sample studied; measure of depression and environment used; genes explored; and GxE findings in relation to these factors.In total, 80% of studies (n = 16) found at least one significant GxE association. However, there was wide variation in methods and analyses adopted across studies, especially with respect to environmental measures used and tests conducted to estimate GxE. This heterogeneity made it difficult to compare findings and evaluate the strength of the evidence for GxE.The existing body of GxE research on depression in youth contains studies that are conceptually and methodologically quite different, which contributes to mixed findings and makes it difficult to assess the current state of the evidence. To decrease this heterogeneity, we offer 20 recommendations that are focused on: (a) reporting GxE research; (b) testing and reporting GxE effects; (c) conceptualizing, measuring and analyzing depression; (d) conceptualizing, measuring and analyzing environment; (e) increasing power to test for GxE; and (f) improving the quality of genetic data used. Although targeted to GxE research on depression, these recommendations can be adopted by GxE researchers focusing on other mental health outcomes.

Project description:BackgroundElectronic health (eHealth) usability evaluations of rapidly developed eHealth systems are difficult to accomplish because traditional usability evaluation methods require substantial time in preparation and implementation. This illustrates the growing need for fast, flexible, and cost-effective methods to evaluate the usability of eHealth systems. To address this demand, the present study systematically identified and expert-validated rapidly deployable eHealth usability evaluation methods.ObjectiveIdentification and prioritization of eHealth usability evaluation methods suitable for agile, easily applicable, and useful eHealth usability evaluations.MethodsThe study design comprised a systematic iterative approach in which expert knowledge was contrasted with findings from literature. Forty-three eHealth usability evaluation methods were systematically identified and assessed regarding their ease of applicability and usefulness through semi-structured expert interviews with 10 European usability experts and systematic literature research. The most appropriate eHealth usability evaluation methods were selected stepwise based on the experts' judgements of their ease of applicability and usefulness.ResultsOf these 43 eHealth usability evaluation methods identified as suitable for agile, easily applicable, and useful eHealth usability evaluations, 10 were recommended by the experts based on their usefulness for rapid eHealth usability evaluations. The three most frequently recommended eHealth usability evaluation methods were Remote User Testing, Expert Review, and Rapid Iterative Test and Evaluation Method. Eleven usability evaluation methods, such as Retrospective Testing, were not recommended for use in rapid eHealth usability evaluations.ConclusionWe conducted a systematic review and expert-validation to identify rapidly deployable eHealth usability evaluation methods. The comprehensive and evidence-based prioritization of eHealth usability evaluation methods supports faster usability evaluations, and so contributes to the ease-of-use of emerging eHealth systems.

Dataset Information

Are university rankings useful to improve research? A systematic review.

Publications

Are university rankings useful to improve research? A systematic review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets