Dataset Information

The impact of Cochrane Systematic Reviews: a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research.

ABSTRACT:

Background

There has been a growing emphasis on evidence-informed decision-making in health care. Systematic reviews, such as those produced by the Cochrane Collaboration, have been a key component of this movement. The UK National Institute for Health Research (NIHR) Systematic Review Programme currently supports 20 Cochrane Review Groups (CRGs). The aim of this study was to identify the impacts of Cochrane reviews published by NIHR-funded CRGs during the years 2007-2011.

Methods

We sent questionnaires to CRGs and review authors, interviewed guideline developers and used bibliometrics and documentary review to get an overview of CRG impact and to evaluate the impact of a sample of 60 Cochrane reviews. We used a framework with four categories (knowledge production, research targeting, informing policy development and impact on practice/services).

Results

A total of 1,502 new and updated reviews were produced by the 20 NIHR-funded CRGs between 2007 and 2011. The clearest impacts were on policy with a total of 483 systematic reviews cited in 247 sets of guidance: 62 were international, 175 national (87 from the UK) and 10 local. Review authors and CRGs provided some examples of impact on practice or services, for example, safer use of medication, the identification of new effective drugs or treatments and potential economic benefits through the reduction in the use of unproven or unnecessary procedures. However, such impacts are difficult to objectively document, and the majority of reviewers were unsure if their review had produced specific impacts. Qualitative data suggested that Cochrane reviews often play an instrumental role in informing guidance, although a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and guideline developers were barriers to their use.

Conclusions

Health and economic impacts of research are generally difficult to measure. We found that to be the case with this evaluation. Impacts on knowledge production and clinical guidance were easier to identify and substantiate than those on clinical practice. Questions remain about how we define and measure impact, and more work is needed to develop suitable methods for impact analysis.

SUBMITTER: Bunn F

PROVIDER: S-EPMC4238314 | biostudies-literature | 2014 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The impact of Cochrane Systematic Reviews: a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research.

Bunn Frances F Trivedi Daksha D Alderson Phil P Hamilton Laura L Martin Alice A Iliffe Steve S

Systematic reviews 20141027

<h4>Background</h4>There has been a growing emphasis on evidence-informed decision-making in health care. Systematic reviews, such as those produced by the Cochrane Collaboration, have been a key component of this movement. The UK National Institute for Health Research (NIHR) Systematic Review Programme currently supports 20 Cochrane Review Groups (CRGs). The aim of this study was to identify the impacts of Cochrane reviews published by NIHR-funded CRGs during the years 2007-2011.<h4>Methods</h4 ...[more]

PMID: 25348511

Similar Datasets

Project description:This research-on-research study describes efforts to develop non-Cochrane systematic reviews (SRs) by analyzing demographical and time-course collaborations between international institutions using protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO) or published in scientific journals. We have published an a priori protocol to develop this study. Protocols published in scientific journals were searched using the MEDLINE and Embase databases; the query terms "Systematic review" [Title] AND "protocol" [Title] were searched from February 2011 to December 2017. Protocols registered at PROSPERO during the same period were obtained by web scraping all non-Cochrane records with a Python script. After excluding protocols that had a fulfillment or duplication rate of less than 90%, they were classified as published "only in PROSPERO", "only in journals", or in "journals and PROSPERO". Results of data and metadata extraction using text mining processes were curated by two reviewers. These Datasets and R scripts are freely available to facilitate reproducibility. We obtained 20,814 protocols of non-Cochrane SRs. While "unique protocols" by reviewers' institutions from 60 countries were the most frequent, a median of 6 (2-150) institutions from 130 different countries were involved in the preparation of "collaborative protocols". The highest Ranked countries involved in overall protocol production were the UK, the U.S., Australia, Brazil, China, Canada, the Netherlands, Germany, and Italy. Most protocols were registered only in PROSPERO. However, the number of protocols published in scientific journals (924) or in both PROSPERO and journals (807) has increased over the last three years. Syst Rev and BMJ Open published more than half of the total protocols. While the more productive countries were involved in "unique" and "collaborative protocols", less productive countries only participated in "collaborative protocols" that were mainly published in PROSPERO. Our results suggest that, although most countries were involved in solitary production of protocols for non-Cochrane SRs during the study period, it would be useful to develop new strategies to promote international collaborations, especially with less productive countries.

Project description:BackgroundThe underrepresentation of women in academic medicine at senior level and in leadership positions is well documented. Biomedical Research Centres (BRC), partnerships between leading National Health Service (NHS) organisations and universities, conduct world class translational research funded by the National Institute for Health Research (NIHR) in the UK. Since 2011 BRCs are required to demonstrate significant progress in gender equity (GE) to be eligible to apply for funding. However, the evidence base for monitoring GE specifically in BRC settings is underdeveloped. This is the first survey tool designed to rank and identify new GE markers specific to the NIHR BRCs.MethodsAn online survey distributed to senior leadership, clinical and non-clinical researchers, trainees, administrative and other professionals affiliated to the NIHR Oxford BRC (N = 683). Participants ranked 13 markers of GE on a five point Likert scale by importance. Data were summarised using frequencies and descriptive statistics. Interrelationships between markers and underlying latent dimensions (factors) were determined by exploratory and confirmatory factor analyses.ResultsThe response rate was 36% (243 respondents). Respondents were more frequently female (55%, n = 133), aged 41-50 years (33%, n = 81), investigators (33%, n = 81) affiliated to the BRC for 2-7 years (39.5%, n = 96). Overall participants ranked 'BRC senior leadership roles' and 'organisational policies on gender equity', to be the most important markers of GE. 58% (n = 141) and 57% (n = 139) respectively. Female participants ranked 'organisational policies' (64.7%, n = 86/133) and 'recruitment and retention' (60.9%, n = 81/133) most highly, whereas male participants ranked 'leadership development' (52.1%, n = 50/96) and 'BRC senior leadership roles' (50%, n = 48/96) as most important. Factor analyses identified two distinct latent dimensions: "organisational markers" and "individual markers" of GE in BRCs.ConclusionsA two-factor model of markers of achievement for GE with "organisational" and "individual" dimensions was identified. Implementation and sustainability of gender equity requires commitment at senior leadership and organisational policy level.

Project description:BackgroundThe burden of pneumonia continues to be substantial, particularly among the poorest in global society. We describe here the trends for UK pneumonia R&D investment and published outputs, and correlate with 2013 global mortality.MethodsData related to awards to UK institutions for pneumonia research from 1997 to 2013 were systematically sourced and categorised by disease area and type of science. Investment was compared to mortality figures in 2010 and 2013 for pneumonia, tuberculosis and influenza. Investment was also compared to publication data.ResultsOf all infectious disease research between 2011 and 2013 (£917.0 million), £28.8 million (3.1%) was for pneumonia. This was an absolute and proportionate increase from previous time periods. Translational pneumonia research (33.3%) received increased funding compared with 1997-2010 where funding was almost entirely preclinical (87.5%, here 30.9%), but high-burden areas such as paediatrics, elderly care and antimicrobial resistance received little investment. Annual investment remains volatile; publication temporal trends show a consistent increase. When comparing investment to global burden with a novel 'investment by mortality observed' metric, tuberculosis (£48.36) and influenza (£484.21) receive relatively more funding than pneumonia (£43.08), despite investment for pneumonia greatly increasing in 2013 compared to 2010 (£7.39). Limitations include a lack of private sector data and the need for careful interpretation of the comparisons with burden, plus categorisation is subjective.ConclusionsThere has been a welcome increase for pneumonia funding awarded to UK institutions in 2011-2013 compared with 1997-2010, along with increases for more translational research. Published outputs relating to pneumonia rose steadily from 1997 to 2013. Investment relative to mortality for pneumonia has increased, but it remains low compared to other respiratory infections and clear inequities remain. Analyses that measure investments in pneumonia can provide an insight into funding trends and research gaps.Research in contextPneumonia continues to be a high-burden illness around the globe. This paper shows that although research funding is increasing in the UK (between 1997 and 2013), it remains poorly funded compared to other important respiratory infectious diseases such as tuberculosis and influenza. Publications about pneumonia have been steadily increasing over time, indicating continuing academic and clinical interest in the topic. Though global mortality of pneumonia is declining, it should still be an area of high priority for funders, policymakers and researchers.

Dataset Information

The impact of Cochrane Systematic Reviews: a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research.

Background

Methods

Results

Conclusions

Publications

The impact of Cochrane Systematic Reviews: a mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets