Dataset Information

Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review.

ABSTRACT:

Background

Prognostic models-used in critical care medicine for mortality predictions, for benchmarking and for illness stratification in clinical trials-have been validated predominantly in high-income countries. These results may not be reproducible in low or middle-income countries (LMICs), not only because of different case-mix characteristics but also because of missing predictor variables. The study objective was to systematically review literature on the use of critical care prognostic models in LMICs and assess their ability to discriminate between survivors and non-survivors at hospital discharge of those admitted to intensive care units (ICUs), their calibration, their accuracy, and the manner in which missing values were handled.

Methods

The PubMed database was searched in March 2017 to identify research articles reporting the use and performance of prognostic models in the evaluation of mortality in ICUs in LMICs. Studies carried out in ICUs in high-income countries or paediatric ICUs and studies that evaluated disease-specific scoring systems, were limited to a specific disease or single prognostic factor, were published only as abstracts, editorials, letters and systematic and narrative reviews or were not in English were excluded.

Results

Of the 2233 studies retrieved, 473 were searched and 50 articles reporting 119 models were included. Five articles described the development and evaluation of new models, whereas 114 articles externally validated Acute Physiology and Chronic Health Evaluation, the Simplified Acute Physiology Score and Mortality Probability Models or versions thereof. Missing values were only described in 34% of studies; exclusion and or imputation by normal values were used. Discrimination, calibration and accuracy were reported in 94.0%, 72.4% and 25% respectively. Good discrimination and calibration were reported in 88.9% and 58.3% respectively. However, only 10 evaluations that reported excellent discrimination also reported good calibration. Generalisability of the findings was limited by variability of inclusion and exclusion criteria, unavailability of post-ICU outcomes and missing value handling.

Conclusions

Robust interpretations regarding the applicability of prognostic models are currently hampered by poor adherence to reporting guidelines, especially when reporting missing value handling. Performance of mortality risk prediction models in LMIC ICUs is at best moderate, especially with limitations in calibration. This necessitates continued efforts to develop and validate LMIC models with readily available prognostic variables, perhaps aided by medical registries.

SUBMITTER: Haniffa R

PROVIDER: S-EPMC5787236 | biostudies-literature | 2018 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background . For the last two decades there has been growing interest in governmental and global health stakeholders about the role that performance measurement and management systems can play for the production of high-quality and safely delivered primary care services. Despite recognition and interest, the gaps in evidence in this field of research and practice in low- and middle-income countries remain poorly characterized. This study will develop an evidence gap map in the area of performance management in primary care delivery systems in low- and middle-income countries. Methods. The evidence gap map will follow the methodology developed by 3Ie, the International Initiative for Impact Evaluation, to systematically map evidence and research gaps. The process starts with the development of the scope by creating an evidence-informed framework that helps identify the interventions and outcomes of relevance as well as help define inclusion and exclusion criteria. A search strategy is then developed to guide the systematic search of the literature, covering the following databases: Medline (Ovid), Embase (Ovid), CAB Global Health (Ovid), CINAHL (Ebsco), Cochrane Library, Scopus (Elsevier), and Econlit (Ovid). Sources of grey literature are also searched. Studies that meet the inclusion criteria are systematically coded, extracting data on intervention, outcome, measures, context, geography, equity, and study design. Systematic reviews are also critically appraised using an existing standard checklist. Impact evaluations are not appraised but will be coded according to study design. The process of map-building ends with the creation of an evidence gap map graphic that displays the available evidence according to the intervention and outcome framework of interest. Discussion . Implications arising from the evidence map will be discussed in a separate paper that will summarize findings and make recommendations for the development of a prioritized research agenda.

Project description:IntroductionPrivate sector healthcare delivery in low- and middle-income countries is sometimes argued to be more efficient, accountable, and sustainable than public sector delivery. Conversely, the public sector is often regarded as providing more equitable and evidence-based care. We performed a systematic review of research studies investigating the performance of private and public sector delivery in low- and middle-income countries.Methods and findingsPeer-reviewed studies including case studies, meta-analyses, reviews, and case-control analyses, as well as reports published by non-governmental organizations and international agencies, were systematically collected through large database searches, filtered through methodological inclusion criteria, and organized into six World Health Organization health system themes: accessibility and responsiveness; quality; outcomes; accountability, transparency, and regulation; fairness and equity; and efficiency. Of 1,178 potentially relevant unique citations, data were obtained from 102 articles describing studies conducted in low- and middle-income countries. Comparative cohort and cross-sectional studies suggested that providers in the private sector more frequently violated medical standards of practice and had poorer patient outcomes, but had greater reported timeliness and hospitality to patients. Reported efficiency tended to be lower in the private than in the public sector, resulting in part from perverse incentives for unnecessary testing and treatment. Public sector services experienced more limited availability of equipment, medications, and trained healthcare workers. When the definition of "private sector" included unlicensed and uncertified providers such as drug shop owners, most patients appeared to access care in the private sector; however, when unlicensed healthcare providers were excluded from the analysis, the majority of people accessed public sector care. "Competitive dynamics" for funding appeared between the two sectors, such that public funds and personnel were redirected to private sector development, followed by reductions in public sector service budgets and staff.ConclusionsStudies evaluated in this systematic review do not support the claim that the private sector is usually more efficient, accountable, or medically effective than the public sector; however, the public sector appears frequently to lack timeliness and hospitality towards patients.

Project description:IntroductionHealth systems responsiveness is a key objective of any health system, yet it is the least studied of all objectives particularly in low-income and middle-income countries. Research on health systems responsiveness highlights its multiple elements, for example, dignity and confidentiality. Little is known, however, about underlying theories of health systems responsiveness, and the mechanisms through which responsiveness works. This realist synthesis contributes to bridging these two knowledge gaps.Methods and analysisIn this realist synthesis, we will use a four-step process, comprising: mapping of theoretical bases, formulation of programme theories, theory refinement and testing of programme theories using literature and empirical data from Ghana and Vietnam. We will include theoretical and conceptual pieces, reviews, empirical studies and grey literature, alongside the primary data. We will explore responsiveness as entailing external and internal interactions within health systems. The search strategy will be purposive and iterative, with continuous screening and refinement of theories. Data extraction will be combined with quality appraisal, using appropriate tools. Each fragment of evidence will be appraised as it is being extracted, for its relevance to the emerging programme theories and methodological rigour. The extracted data pertaining to contexts, mechanisms and outcomes will be synthesised to identify patterns and contradictions. Results will be reported using narrative explanations, following established guidance on realist syntheses.Ethics and disseminationEthics approvals for the wider RESPONSE (Improving health systems responsiveness to neglected health needs of vulnerable groups in Ghana and Vietnam) study, of which this review is one part, were obtained from the ethics committees of the following institutions: London School of Hygiene and Tropical Medicine (ref: 22981), University of Leeds, School of Medicine (ref: MREC19-051), Ghana Health Service (ref: GHS-ERC 012/03/20) and Hanoi University of Public Health (ref: 020-149/DD-YTCC).We will disseminate results through academic papers, conference presentations and stakeholder workshops in Ghana and Vietnam.Prospero registration numberCRD42020200353. Full record: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020200353.

Project description:System dynamics methods were used to explore effective implementation pathways for improving health systems performance through pay-for-performance (P4P) schemes. A causal loop diagram was developed to delineate primary causal relationships for service delivery within primary health facilities. A quantitative stock-and-flow model was developed next. The stock-and-flow model was then used to simulate the impact of various P4P implementation scenarios on quality and volume of services. Data from the Afghanistan national facility survey in 2012 was used to calibrate the model. The models show that P4P bonuses could increase health workers' motivation leading to higher levels of quality and volume of services. Gaming could reduce or even reverse this desired effect, leading to levels of quality and volume of services that are below baseline levels. Implementation issues, such as delays in the disbursement of P4P bonuses and low levels of P4P bonuses, also reduce the desired effect of P4P on quality and volume, but they do not cause the outputs to fall below baseline levels. Optimal effect of P4P on quality and volume of services is obtained when P4P bonuses are distributed per the health workers' contributions to the services that triggered the payments. Other distribution algorithms such as equal allocation or allocations proportionate to salaries resulted in quality and volume levels that were substantially lower, sometimes below baseline. The system dynamics models served to inform, with quantitative results, the theory of change underlying P4P intervention. Specific implementation strategies, such as prompt disbursement of adequate levels of performance bonus distributed per health workers' contribution to service, increase the likelihood of P4P success. Poorly designed P4P schemes, such as those without an optimal algorithm for distributing performance bonuses and adequate safeguards for gaming, can have a negative overall impact on health service delivery systems.