Dataset Information

Investigating the nature and quality of locally commissioned evaluations of the NHS Vanguard programme: an evidence synthesis.

ABSTRACT:

Background

With innovation in service delivery increasingly viewed as crucial to the long-term sustainability of health systems, NHS England launched an ambitious new model of care (Vanguard) programme in 2015. Supported by a £350 million transformation fund, 50 Vanguard sites were to act as pilots for innovation in service delivery, to move quickly to change the way that services were delivered, breaking down barriers between sectors and improving the coordination and delivery of care.

Methods

As part of a national evaluation of the Vanguard programme, we conducted an evidence synthesis to assess the nature and quality of locally commissioned evaluations. With access to a secure, online hub used by the Vanguard and other integrated care initiatives, two researchers retrieved any documents from a locally commissioned evaluation for inclusion. All identified documents were downloaded and logged, and details of the evaluators, questions, methodological approaches and limitations in design and/or reporting were extracted. As included evaluations varied in nature and type, a narrative synthesis was undertaken.

Results

We identified a total of 115 separate reports relating to the locally commissioned evaluations. Five prominent issues relating to evaluation conduct were identified across included reports: use of logic models, number and type of evaluation questions posed, data sharing and information governance, methodological challenges and evaluation reporting in general. A combination of resource, data and time constraints means that evaluations often attempted to but did not fully address the wide range of questions posed by individual Vanguards.

Conclusions

Significant investment was made in independent local evaluations of the Vanguard programme by NHS England. This synthesis represents the only comprehensive attempt to capture methodological learning and may serve as a key resource for researchers and policy-makers seeking to understand investigating large-scale system change, both within the NHS and internationally. PROSPERO (Registration number: CRD42017069282).

SUBMITTER: Wilson P

PROVIDER: S-EPMC8042862 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo update a rapid review published in 2017, which evaluated the NHS Health Check programme.MethodsAn enlarged body of evidence was used to readdress six research objectives from a rapid review published in 2017, relating to the uptake, patient experiences and effectiveness of the NHS Health Check programme. Data sources included MEDLINE, PubMed, Embase, Health Management Information Consortium (HMIC), Cumulative Index of Nursing and Allied Health Literature (CINAHL), Global Health, PsycINFO, the Cochrane Library, NHS Evidence, Google Scholar, Google, ClinicalTrials.gov and the ISRCTN registry, Web of Science, Science Citation Index, The Cochrane Library, NHS Evidence, OpenGrey and hand searching article reference lists. These searches identified records from between January 1996 and December 2019. Screening, data extraction and quality appraisal using the Critical Appraisals Skills Programme checklists were performed in duplicate. Grading of Recommendations Assessment, Development and Evaluations was implemented. Data were synthesised narratively.Results697 studies were identified, and 29 new studies included in the review update. The number of published studies on the uptake, patient experiences and effectiveness of the NHS Health Check programme has increased by 43% since the rapid review published in 2017. However, findings from the original review remain largely unchanged. NHS Health Checks led to an overall increase in the detection of raised risk factors and morbidities including diabetes mellitus, hypertension, raised blood pressure, cholesterol and chronic kidney disease. Individuals most likely to attend the NHS Health Check programme included women, persons aged ≥60 years and those from more socioeconomically advantaged backgrounds. Opportunistic invitations increased uptake among men, younger persons and those with a higher deprivation level.ConclusionsAlthough results are inconsistent between studies, the NHS Health Check programme is associated with increased detection of heightened cardiovascular disease risk factors and diagnoses. Uptake varied between population subgroups. Opportunistic invitations may increase uptake.

Project description:BackgroundThe nationwide integration of palliative care best practices into general care settings is challenging but important in improving the quality of palliative care. This is why the Dutch National Quality Improvement Programme for Palliative Care has recently been launched. This four-year programme consists of about 70 implementation trajectories of best practices. A large evaluation study has been set up to evaluate this national programme and separate implementation trajectories.Methods/designThis paper presents the protocol of the evaluation study consisting of a quantitative effect evaluation and a qualitative process evaluation. The effect evaluation has a pre-test post-test design, with measurements before implementation (month 0) and after implementation (month 9) of a best practice. Patients are eligible if they have a life expectancy of less than six months and/or if they are undergoing palliative treatment and provided they are physically and mentally capable of responding to questionnaires. Bereaved relatives are eligible if they have been involved in the care of a deceased patient who died after a sickbed between six weeks and six months ago. Three types of measurement instruments are used: (1) numerical rating scales for six symptoms (pain, fatigue, breathlessness, obstipation, sadness and anxiety), (2) the Consumer Quality Index Palliative Care - patient version and (3) the version for bereaved relatives.The process evaluation consists of analysing implementation plans and reports of the implementation, and individual and group interviews with healthcare professionals. This will be done nine to eleven months after the start of the implementation of a best practice.DiscussionThis mixed-method evaluation study gives more insight into the effects of the total programme and the separate implementation trajectories. However, evaluation of large quality improvement programmes is complicated due to changing, non-controlled environments. Therefore, it is important that an effect evaluation is combined with a process evaluation.Trial registrationNTR-4085.

Project description:IntroductionThe NHS Health Check aims to identify individuals at increased risk of cardiovascular diseases (CVDs) among the adult population in England. The Health Check includes calculation of CVD risk and discussion of pharmacological and lifestyle approaches to manage risk, including referral to lifestyle support services. The programme is commissioned by Local Authorities (LAs) and is delivered by a range of different providers in different settings. There is significant variation in activity, with uptake ranging from 25% to 85% in different areas, and clear evidence of variation in implementation and delivery practice.Methods and analysisWe aim to understand how the NHS Health Check programme works in different settings, for different groups, so that we can recommend improvements to maximise intended outcomes. To do so, we will undertake a realist review and a survey of LA public health teams. Our review will follow Pawson's five iterative stages: (1) locate existing theories, (2) search for evidence, (3) article selection, (4) extract and organise data and (5) synthesise evidence and draw conclusions. Our review will include documents describing local implementation alongside published research studies. We will recruit a stakeholder group (including Public Health England, commissioners and providers of Health Checks, plus members of the public and patients) to advise us throughout. Our survey will be sent to all 152 LAs in England to gather detailed information on programme delivery (including COVID-19-related changes) and available referral services. This will enable us to map delivery across England and relate these data to programme outcomes.Ethics and disseminationEthical approval is not required for this review. For the survey, we have received approval from the University of Kent Research Ethics Committee. Our findings will be used to develop recommendations on tailoring, implementation and design strategies to improve delivery of the NHS Health Check in different settings, for different groups.Prospero registration numberCRD42020163822.

Project description:OBJECTIVES:Evaluation of the demonstrator phase and first wave roll-out of the National Health Service (NHS) Diabetes Prevention Programme (DPP) in England. To examine: (1) intervention design, provision and fidelity assessment procedures; (2) risk assessment and recruitment pathways and (3) data collection for monitoring and evaluation. To provide recommendations informing decision makers on programme quality, improvements and future evaluation. DESIGN:We reviewed programme documents, mapping against the NHS DPP specification and National Institute for Health and Care Excellence (NICE) public health guideline: Type 2 diabetes (T2D) prevention in people at high risk (PH38), conducted qualitative research using individual interviews and focus group discussions with stakeholders and examined recruitment, fidelity and data collection procedures. SETTING:Seven NHS DPP demonstrator sites and, subsequently, 27 first wave areas across England. INTERVENTIONS:Intensive behavioural intervention with weight loss, diet and physical activity goals. The national programme specifies at least 13 sessions over 9?months, delivered face to face to groups of 15-20 adults with non-diabetic hyperglycaemia, mainly recruited from primary care and NHS Health Checks. PARTICIPANTS:Participants for qualitative research were purposively sampled to provide a spread of stakeholder experience. Documents for review were provided via the NHS DPP Management Group. FINDINGS:The NHS DPP specification reflected current evidence with a clear framework for service provision. Providers, with national capacity to deliver, supplied intervention plans compliant with this framework. Stakeholders highlighted limitations in fidelity assessment and recruitment and retention challenges, especially in reach and equity, that could adversely impact on implementation. Risk assessment for first wave eligibility differed from NICE guidance. CONCLUSIONS:The NHS DPP provides an evidence-based behavioural intervention for prevention of T2D in adults at high risk, with capacity to deliver nationally. Framework specification allows for balance between consistency and contextual variation in intervention delivery, with session details devolved to providers. Limitations in fidelity assurance, data collection procedures and recruitment issues could adversely impact on intervention effectiveness and restrict evaluation.