Dataset Information

Use of concurrent evaluation to improve implementation of a home fortification programme in Bangladesh: a methodological innovation.

ABSTRACT:

Objective

This paper focuses on the use of 'concurrent evaluation' to evaluate a nationally scaled-up programme in Bangladesh that was implemented by BRAC (an international development organisation) using Shasthya Shebika (SS) - volunteer community health workers - to promote home fortification with micronutrient powders (MNP) for children under-five.

Design

We developed a programme impact pathway to conceptualise the implementation and evaluation strategy and developed a strategic partnership among the key programme stakeholders for better use of evaluation evidence. We developed a multi-method concurrent evaluation strategy to provide insights into the BRAC programme and created provision for course correction to the implementation plan while it was in operation.

Setting

One hundred sixty-four sub-districts and six urban slums in Bangladesh.

Participants

Caregivers of children 6-59 months, SS and BRAC's staff members.

Results

The evaluation identified low awareness about home fortification among caregivers, inadequate supply and frequent MNP stockouts, and inadequate skills of BRAC's SS to promote MNP at the community level as hindrances to the achievement of programme goals. The partners regularly discussed evaluation results during and after implementation activities to assess progress in programme coverage and any needs for modification. BRAC initiated a series of corrections to the original implementation plan to address these challenges, which improved the design of the MNP programme; this resulted in enhanced programme outcomes.

Conclusions

Concurrent evaluation is an innovative approach to evaluate complex real-world programmes. Here it was utilised in implementing a large-scale nutrition programme to measure implementation process and effectiveness.

SUBMITTER: Sarma H

PROVIDER: S-EPMC8042578 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundHome visiting programs are implemented in high income countries to improve outcomes for families with young children. Significant resources are invested in such programs and high quality evaluations are important. In the context of research trials, implementation quality is often poorly reported and, when reported, is variable. This paper presents the quality of implementation of the right@home program, a sustained nurse home visiting intervention trialled in Australia, and delivered in a 'real world' context through usual child and family health services. right@home is structured around the core Maternal Early Childhood Sustained Home-visiting (MECSH) program, which is a salutogenic, child focused prevention model.MethodAt each visit right@home practitioners completed a checklist detailing the client unique identifier, date of contact and activities undertaken. These checklists were collated to provide data on intervention dose, retention to program completion at child age 2 years, and visit content, which were compared with the program schedule. Quality of family-provider relationship was measured using the Session Rating Scale. Exploratory factor analysis was conducted to identify clusters of activities and allow qualitative assessment of concordance between program aims and program delivery.ResultsOf 363 intervention families offered the program, 352 (97·0%) commenced the program and 304 (87·3%) completed the program to child age 2 years. 253 of 352 (71·9%) families who commenced the program received more than 75 percent of scheduled visits including at least one antenatal visit. Families rated the participant-practitioner relationship highly (mean 39.4/40). The factor analysis identified six antenatal and six postnatal components which were concordant with the program aims.ConclusionsThe right@home program was delivered with higher adherence to program dose, schedule and content, and retention than usually reported in other home visiting research. Program compliance may have resulted from program design (visit schedule, dose, content and delivery flexibility) that was consistent with family aims.

Project description:BackgroundAnemia poses a significant public health problem, affecting 1.6 billion people and contributing to the loss of 68.4 million disability-adjusted life years. We assessed the impact of a market-based home fortification program with micronutrient powder (MNP) called Pushtikona-5 implemented by Bangladesh Rural Advancement Committee (BRAC) on the prevalence of anemia among children aged 6-59 months in Bangladesh.MethodsWe used a modified stepped wedged design and conducted three baseline, two midline, and three endline surveys to evaluate the Pushtikona-5 program implemented through three BRAC program platforms. We interviewed children's caregivers, and collected finger-prick blood samples from children to measure hemoglobin concentration. We also collected data on coverage of Pushtikona-5 and infant and young child feeding (IYCF) practices. We performed bivariate and multivariable analysis and calculated adjusted risk ratios (ARRs) to assess the effect of program outcomes.ResultsA total of 16,936 households were surveyed. The prevalence of anemia was 46.6% at baseline, dropping to 32.1% at midline and 31.2% at endline. These represented adjusted relative reductions of 34% at midline (RR 0.66, 95%CI 0.62 to 0.71, value of p <0.001) and 32% at endline (RR 0.68, 95%CI 0.64 to 0.71, value of p <0.001) relative to baseline. Regarding MNP coverage, at baseline, 43.5% of caregivers surveyed had heard about MNP; 24.3% of children had ever consumed food with MNP, and only 1.8% had consumed three or more sachets in the 7 days preceding the survey. These increased to 63.0, 36.9, and 4.6%, respectively, at midline and 90.6, 68.9, and 11.5%, respectively, at endline.ConclusionThese results show evidence of a reduction in the prevalence of anemia and an improvement in coverage. This study provides important evidence of the feasibility and potential for impact of linking market-based MNP distribution with IYCF promotion through community level health workers.

Project description:BackgroundOver the last three decades in response to the growing burden of diabetes, countries worldwide have developed national and regional multifaceted programmes to improve the monitoring and management of diabetes and to enhance the coordination of care within and across settings. In Ireland in 2010, against a backdrop of limited dedicated strategic planning and engrained variation in the type and level of diabetes care, a national programme was established to standardise and improve care for people with diabetes in Ireland, known as the National Diabetes Programme (NDP). The NDP comprises a range of organisational and service delivery changes to support evidence-based practices and policies. This realist evaluation protocol sets out the approach that will be used to identify and explain which aspects of the programme are working, for whom and in what circumstances to produce the outcomes intended.Methods/designThis mixed method realist evaluation will develop theories about the relationship between the context, mechanisms and outcomes of the diabetes programme. In stage 1, to identify the official programme theories, documentary analysis and qualitative interviews were conducted with national stakeholders involved in the design, development and management of the programme. In stage 2, as part of a multiple case study design with one case per administrative region in the health system, qualitative interviews are being conducted with frontline staff and service users to explore their responses to, and reasoning about, the programme's resources (mechanisms). Finally, administrative data will be used to examine intermediate implementation outcomes such as service uptake, acceptability, and fidelity to models of care.DiscussionThis evaluation is using the principles of realist evaluation to examine the implementation of a national programme to standardise and improve services for people with diabetes in Ireland. The concurrence of implementation and evaluation has enabled us to produce formative feedback for the NDP while also supporting the refinement and revision of initial theories about how the programme is being implemented in the dynamic and unstable context of the Irish healthcare system.

Project description:OBJECTIVES:Evaluation of the demonstrator phase and first wave roll-out of the National Health Service (NHS) Diabetes Prevention Programme (DPP) in England. To examine: (1) intervention design, provision and fidelity assessment procedures; (2) risk assessment and recruitment pathways and (3) data collection for monitoring and evaluation. To provide recommendations informing decision makers on programme quality, improvements and future evaluation. DESIGN:We reviewed programme documents, mapping against the NHS DPP specification and National Institute for Health and Care Excellence (NICE) public health guideline: Type 2 diabetes (T2D) prevention in people at high risk (PH38), conducted qualitative research using individual interviews and focus group discussions with stakeholders and examined recruitment, fidelity and data collection procedures. SETTING:Seven NHS DPP demonstrator sites and, subsequently, 27 first wave areas across England. INTERVENTIONS:Intensive behavioural intervention with weight loss, diet and physical activity goals. The national programme specifies at least 13 sessions over 9?months, delivered face to face to groups of 15-20 adults with non-diabetic hyperglycaemia, mainly recruited from primary care and NHS Health Checks. PARTICIPANTS:Participants for qualitative research were purposively sampled to provide a spread of stakeholder experience. Documents for review were provided via the NHS DPP Management Group. FINDINGS:The NHS DPP specification reflected current evidence with a clear framework for service provision. Providers, with national capacity to deliver, supplied intervention plans compliant with this framework. Stakeholders highlighted limitations in fidelity assessment and recruitment and retention challenges, especially in reach and equity, that could adversely impact on implementation. Risk assessment for first wave eligibility differed from NICE guidance. CONCLUSIONS:The NHS DPP provides an evidence-based behavioural intervention for prevention of T2D in adults at high risk, with capacity to deliver nationally. Framework specification allows for balance between consistency and contextual variation in intervention delivery, with session details devolved to providers. Limitations in fidelity assurance, data collection procedures and recruitment issues could adversely impact on intervention effectiveness and restrict evaluation.

Project description:IntroductionCardiac rehabilitation (CR) improves health-related quality of life and reduces hospital admissions. However, patients with heart failure (HF) often fail to attend centre-based CR programmes. Novel ways of delivering healthcare, such as home-based CR programmes, may improve uptake of CR. Rehabilitation EnAblement in CHronic Heart Failure (REACH-HF) is a new, effective and cost-effective home-based CR programme for people with HF. The aim of this prospective mixed-method implementation evaluation study is to assess the implementation of the REACH-HF CR programme in the UK National Health Service (NHS). The specific objectives are to (1) explore NHS staff perceptions of the barriers and facilitators to the implementation of REACH-HF, (2) assess the quality of delivery of the programme in real-life clinical settings, (3) consider the nature of any adaptation(s) made and how they might impact on intervention effectiveness and (4) compare real-world patient outcomes to those seen in a prior clinical trial.Methods and analysisREACH-HF will be rolled out in four NHS CR centres across the UK. Three healthcare professionals from each site will be trained to deliver the 12-week programme. In-depth qualitative interviews and focus groups will be conducted with approximately 24 NHS professionals involved in delivering or commissioning the programme. Consultations for 48 patients (12 per site) will be audio recorded and scored using an intervention fidelity checklist. Outcomes routinely recorded in the National Audit of Cardiac Rehabilitation will be analysed and compared with outcomes from a recent randomised controlled trial: the Minnesota Living with HF Questionnaire and exercise capacity (Incremental Shuttle Walk Test). Qualitative research findings will be mapped onto the Normalisation Process Theory framework and presented in the form of a narrative synthesis. Results of the study will inform national roll-out of REACH-HF.Ethics and disseminationThe study (IRAS 261723) has received ethics approval from the South Central (Hampshire B) Research Ethics Committee (19/SC/0304). Written informed consent will be obtained from all health professionals and patients participating in the study. The research team will ensure that the study is conducted in accordance with the Declaration of Helsinki, the Data Protection Act 2018, General Data Protection Regulations and in accordance with the Research Governance Framework for Health and Social Care (2005). Findings will be published in scientific peer-reviewed journals and presented at local, national and international meetings to publicise and explain the research methods and findings to key audiences to facilitate the further uptake of the REACH-HF intervention.Trial registrationISRCTN86234930.