Dataset Information

Opening the 'black box' of collaborative improvement: a qualitative evaluation of a pilot intervention to improve quality of malaria surveillance data in public health centres in Uganda.

ABSTRACT:

Background

Demand for high-quality surveillance data for malaria, and other diseases, is greater than ever before. In Uganda, the primary source of malaria surveillance data is the Health Management Information System (HMIS). However, HMIS data may be incomplete, inaccurate or delayed. Collaborative improvement (CI) is a quality improvement intervention developed in high-income countries, which has been advocated for low-resource settings. In Kayunga, Uganda, a pilot study of CI was conducted in five public health centres, documenting a positive effect on the quality of HMIS and malaria surveillance data. A qualitative evaluation was conducted concurrently to investigate the mechanisms of effect and unintended consequences of the intervention, aiming to inform future implementation of CI.

Methods

The study intervention targeted health workers, including brief in-service training, plus CI with 'plan-do-study-act' (PDSA) cycles emphasizing self-reflection and group action, periodic learning sessions, and coaching from a CI mentor. Health workers collected data on standard HMIS out-patient registers. The qualitative evaluation (July 2015 to September 2016) included ethnographic observations at each health centre (over 12-14 weeks), in-depth interviews with health workers and stakeholders (n = 20), and focus group discussions with health workers (n = 6).

Results

The results suggest that the intervention did facilitate improvement in data quality, but through unexpected mechanisms. The CI intervention was implemented as planned, but the PDSA cycles were driven largely by the CI mentor, not the health workers. In this context, characterized by a rigid hierarchy within the health system of limited culture of self-reflection and inadequate training and supervision, CI became an effective form of high-quality training with frequent supervisory visits. Health workers appeared motivated to improve data collection habits by their loyalty to the CI mentor and the potential for economic benefits, rather than a desire for self-improvement.

Conclusions

CI is a promising method of quality improvement and could have a positive impact on malaria surveillance data. However, successful scale-up of CI in similar settings may require deployment of highly skilled mentors. Further research, focusing on the effectiveness of 'real world' mentors using robust study designs, will be required to determine whether CI can be translated effectively and sustainably to low-resource settings.

SUBMITTER: Hutchinson E

PROVIDER: S-EPMC8243860 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundSurveillance data are essential for malaria control, but quality is often poor. The aim of the study was to evaluate the effectiveness of the novel combination of training plus an innovative quality improvement method-collaborative improvement (CI)-on the quality of malaria surveillance data in Uganda.MethodsThe intervention (training plus CI, or TCI), including brief in-service training and CI, was delivered in 5 health facilities (HFs) in Kayunga District from November 2015 to August 2016. HF teams monitored data quality, conducted plan-do-study-act cycles to test changes, attended periodic learning sessions, and received CI coaching. An independent evaluation was conducted to assess data completeness, accuracy, and timeliness. Using an interrupted time series design without a separate control group, data were abstracted from 156,707 outpatient department (OPD) records, laboratory registers, and aggregated monthly reports (MR) for 4 time periods: baseline-12 months, TCI scale-up-5 months; CI implementation-9 months; post-intervention-4 months. Monthly OPD register completeness was measured as the proportion of patient records with a malaria diagnosis with: (1) all data fields completed, and (2) all clinically-relevant fields completed. Accuracy was the relative difference between: (1) number of monthly malaria patients reported in OPD register versus MR, and (2) proportion of positive malaria tests reported in the laboratory register versus MR. Data were analysed with segmented linear regression modelling.ResultsData completeness increased substantially following TCI. Compared to baseline, all-field completeness increased by 60.1%-points (95% confidence interval [CI]: 46.9-73.2%) at mid-point, and clinically-relevant completeness increased by 61.6%-points (95% CI: 56.6-66.7%). A relative - 57.4%-point (95% confidence interval: - 105.5, - 9.3%) change, indicating an improvement in accuracy of malaria test positivity reporting, but no effect on data accuracy for monthly malaria patients, were observed. Cost per additional malaria patient, for whom complete clinically-relevant data were recorded in the OPD register, was $3.53 (95% confidence interval: $3.03, $4.15).ConclusionsTCI improved malaria surveillance completeness considerably, with limited impact on accuracy. Although these results are promising, the intervention's effectiveness should be evaluated in more HFs, with longer follow-up, ideally in a randomized trial, before recommending CI for wide-scale use.

Project description:In Uganda, health system challenges limit access to good quality healthcare and contribute to slow progress on malaria control. We developed a complex intervention (PRIME), which was designed to improve quality of care for malaria at public health centres.Responding to calls for increased transparency, we describe the PRIME intervention's design process, rationale, and final content and reflect on the choices and challenges encountered during the design of this complex intervention.To develop the intervention, we followed a multistep approach, including the following: 1) formative research to identify intervention target areas and objectives; 2) prioritization of intervention components; 3) review of relevant evidence; 4) development of intervention components; 5) piloting and refinement of workshop modules; and 6) consolidation of the PRIME intervention theories of change to articulate why and how the intervention was hypothesized to produce desired outcomes. We aimed to develop an intervention that was evidence-based, grounded in theory, and appropriate for the study context; could be evaluated within a randomized controlled trial; and had the potential to be scaled up sustainably.The process of developing the PRIME intervention package was lengthy and dynamic. The final intervention package consisted of four components: 1) training in fever case management and use of rapid diagnostic tests for malaria (mRDTs); 2) workshops in health centre management; 3) workshops in patient-centred services; and 4) provision of mRDTs and antimalarials when stocks ran low.The slow and iterative process of intervention design contrasted with the continually shifting study context. We highlight the considerations and choices made at each design stage, discussing elements we included and why, as well as those that were ultimately excluded. Reflection on and reporting of 'behind the scenes' accounts of intervention design may improve the design, assessment, and generalizability of complex interventions and their evaluations.

Project description:HIV and malaria exert co-pathogenic effects. Malaria surveillance data are necessary for public health strategies to reduce the burden of disease in high HIV prevalence settings.This was a longitudinal cohort study to assess the burden of malaria in rural Rakai, Uganda. Households were visited monthly for 1 year to identify confirmed clinical malaria (CCM), or parasitaemia with temperature >37.5 °C, and asymptomatic parasitaemia (AP). Interviews of the adult or child's caregiver and clinical and laboratory assessments were conducted. Rapid diagnostic testing for malaria and anaemia was performed if participants were febrile and anti-malarial treatment given per Uganda Ministry of Health 2010 guidelines. Blood was drawn at every household visit to assess for parasitaemia, and blood smears were assessed at the Rakai Health Science Programme laboratory.A total of 1640 participants were enrolled, including 975 children aged 6 months up to 10 years, 393 adult caregivers, and 272 adolescent/adult household members from 393 randomly selected households in two representative communities. 1459 (89 %) participants completed all study visits. CCM was identified in 304 (19 %) participants, with the highest incidence rate for CCM of 0.38 per person-year (ppy) identified in children <5 years, and rates decreased with age; the rates were 0.27, 0.16, and 0.09 ppy for ages 5-<10 years, 10-<18 years, and adults 18+ years, respectively. AP was identified in 943 (57 %) participants; the incidence rate was 1.99 ppy for <5 years, 2.72 ppy for 5-<10 years, 2.55 ppy for 10-<18 years, and 0.86 ppy among adults, with 92 % of cases being attributed to Plasmodium falciparum by smear. 994 (61 %) individuals had at least one positive smear; 342 (21 %) had one positive result, 203 (12 %) had two, 115 (7 %) had three, and 334 (21 %) had >3 positive smears during follow-up. Seasonal rates generally followed the rains and peaked during July, then decreased through November before increasing again.Plasmodium falciparum infection remains high in rural Uganda. Increased malaria control interventions should be prioritized. Trial registration Clinicaltrials.gov identifier NCT01265407.

Project description:BackgroundAccurate measures of malaria incidence are essential to track progress and target high-risk populations. While health management information system (HMIS) data provide counts of malaria cases, quantifying the denominator for incidence using these data is challenging because catchment areas and care-seeking behaviours are not well defined. This study's aim was to estimate malaria incidence using HMIS data by adjusting the population denominator accounting for travel time to the health facility.MethodsOutpatient data from two public health facilities in Uganda (Kihihi and Nagongera) over a 3-year period (2011-2014) were used to model the relationship between travel time from patient village of residence (available for each individual) to the facility and the relative probability of attendance using Poisson generalized additive models. Outputs from the model were used to generate a weighted population denominator for each health facility and estimate malaria incidence. Among children aged 6 months to 11 years, monthly HMIS-derived incidence estimates, with and without population denominators weighted by probability of attendance, were compared with gold standard measures of malaria incidence measured in prospective cohorts.ResultsA total of 48,898 outpatient visits were recorded across the two sites over the study period. HMIS incidence correlated with cohort incidence over time at both study sites (correlation in Kihihi = 0.64, p < 0.001; correlation in Nagongera = 0.34, p = 0.045). HMIS incidence measures with denominators unweighted by probability of attendance underestimated cohort incidence aggregated over the 3 years in Kihihi (0.5 cases per person-year (PPY) vs 1.7 cases PPY) and Nagongera (0.3 cases PPY vs 3.0 cases PPY). HMIS incidence measures with denominators weighted by probability of attendance were closer to cohort incidence, but remained underestimates (1.1 cases PPY in Kihihi and 1.4 cases PPY in Nagongera).ConclusionsAlthough malaria incidence measured using HMIS underestimated incidence measured in cohorts, even when adjusting for probability of attendance, HMIS surveillance data are a promising and scalable source for tracking relative changes in malaria incidence over time, particularly when the population denominator can be estimated by incorporating information on village of residence.

Project description:State-of-the-art machine learning (ML) artificial intelligence methods are increasingly leveraged in clinical predictive modeling to provide clinical decision support systems to physicians. Modern ML approaches such as artificial neural networks (ANNs) and tree boosting often perform better than more traditional methods like logistic regression. On the other hand, these modern methods yield a limited understanding of the resulting predictions. However, in the medical domain, understanding of applied models is essential, in particular, when informing clinical decision support. Thus, in recent years, interpretability methods for modern ML methods have emerged to potentially allow explainable predictions paired with high performance. To our knowledge, we present in this work the first explainability comparison of two modern ML methods, tree boosting and multilayer perceptrons (MLPs), to traditional logistic regression methods using a stroke outcome prediction paradigm. Here, we used clinical features to predict a dichotomized 90 days post-stroke modified Rankin Scale (mRS) score. For interpretability, we evaluated clinical features' importance with regard to predictions using deep Taylor decomposition for MLP, Shapley values for tree boosting and model coefficients for logistic regression. With regard to performance as measured by Area under the Curve (AUC) values on the test dataset, all models performed comparably: Logistic regression AUCs were 0.83, 0.83, 0.81 for three different regularization schemes; tree boosting AUC was 0.81; MLP AUC was 0.83. Importantly, the interpretability analysis demonstrated consistent results across models by rating age and stroke severity consecutively amongst the most important predictive features. For less important features, some differences were observed between the methods. Our analysis suggests that modern machine learning methods can provide explainability which is compatible with domain knowledge interpretation and traditional method rankings. Future work should focus on replication of these findings in other datasets and further testing of different explainability methods.