Dataset Information

Systematic approach to evaluating and confirming the utility of a suite of national health system performance (HSP) indicators in Canada: a modified Delphi study.

ABSTRACT:

Objectives

Evaluating an existing suite of health system performance (HSP) indicators for continued reporting using a systematic criteria-based assessment and national consensus conference.

Design

Modified Delphi approach with technical and leadership groups, an online survey of stakeholders and convening a national consensus conference.

Setting

A national health information steward, the Canadian Institute for Health Information (CIHI).

Participants

A total of 73 participants, comprised 61 conference attendants/stakeholders from across Canada and 12 national health information steward staff.

Primary and secondary outcome measures

Indicator dispositions of retention, additional stakeholder consultation, further redevelopment or retirement.

Results

4 dimensions (usability, importance, scientific soundness and feasibility) typically used to select measures for reporting were expanded to 18 criteria grouped under the 4 dimensions through a process of research and testing. Definitions for each criterion were developed and piloted. Once the definitions were established, 56 of CIHI's publicly reported HSP indicators were evaluated against the criteria using modified Delphi approaches. Of the 56 HSP indicators evaluated, 9 measures were ratified for retirement, 7 were identified for additional consultation and 3 for further research and development. A pre-Consensus Conference survey soliciting feedback from stakeholders on indicator recommendations received 48 responses (response rate of 79%).

Conclusions

A systematic evaluation of HSP indicators informed the development of objective recommendations for continued reporting. The evaluation was a fruitful exercise to identify technical considerations for calculating indicators, furthering our understanding of how measures are used by stakeholders, as well as harmonising actions that could be taken to ensure relevancy, reduce indicator chaos and build consensus with stakeholders.

SUBMITTER: Fekri O

PROVIDER: S-EPMC5775456 | biostudies-literature | 2017 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Systematic approach to evaluating and confirming the utility of a suite of national health system performance (HSP) indicators in Canada: a modified Delphi study.

Fekri Omid O Leeb Kira K Gurevich Yana Y

BMJ open 20170412 4

<h4>Objectives</h4>Evaluating an existing suite of health system performance (HSP) indicators for continued reporting using a systematic criteria-based assessment and national consensus conference.<h4>Design</h4>Modified Delphi approach with technical and leadership groups, an online survey of stakeholders and convening a national consensus conference.<h4>Setting</h4>A national health information steward, the Canadian Institute for Health Information (CIHI).<h4>Participants</h4>A total of 73 par ...[more]

PMID: 28404612

Similar Datasets

Project description:BackgroundIndicators are essential instruments for monitoring and evaluating population health. The selection of a multidimensional set of indicators should not only reflect the scientific evidence on health outcomes and health determinants, but also the views of health experts and stakeholders. The aim of this study is to describe the Delphi selection process designed to promote agreement on indicators considered relevant to evaluate population health at the European regional level.MethodsIndicators were selected in a Delphi survey conducted using a web-platform designed to implement and monitor participatory processes. It involved a panel of 51 experts and 30 stakeholders from different areas of knowledge and geographies. In three consecutive rounds the panel indicated their level of agreement or disagreement with indicator's relevance for evaluating population health in Europe. Inferential statistics were applied to draw conclusions on observed level of agreement (Scott's Pi interrater reliability coefficient) and opinion change (McNemar Chi-square test). Multivariate analysis of variance was conducted to check if the field of expertise influenced the panellist responses (Wilk's Lambda test).ResultsThe panel participated extensively in the study (overall response rate: 80%). Eighty indicators reached group agreement for selection in the areas of: economic and social environment (12); demographic change (5); lifestyle and health behaviours (8); physical environment (6); built environment (12); healthcare services (11) and health outcomes (26). Higher convergence of group opinion towards agreement on the relevance of indicators was seen for lifestyle and health behaviours, healthcare services, and health outcomes. The panellists' field of expertise influenced responses: statistically significant differences were found for economic and social environment (p?<?0.05 in round 1 and 2), physical environment (p?<?0.01 in round 1) and health outcomes (p?<?0.01 in round 3).ConclusionsThe high levels of participation observed in this study, by involving experts and stakeholders and ascertaining their views, underpinned the added value of using a transparent Web-Delphi process to promote agreement on what indicators are relevant to appraise population health.

Project description:BackgroundThe detection and management of chronic kidney disease lies within primary care; however, performance measures applicable in the Canadian context are lacking. We sought to develop a set of primary care quality indicators for chronic kidney disease in the Canadian setting and to assess the current state of the disease's detection and management in primary care.MethodsWe used a modified Delphi panel approach, involving 20 panel members from across Canada (10 family physicians, 7 nephrologists, 1 patient, 1 primary care nurse and 1 pharmacist). Indicators identified from peer-reviewed and grey literature sources were subjected to 3 rounds of voting to develop a set of quality indicators for the detection and management of chronic kidney disease in the primary care setting. The final indicators were applied to primary care electronic medical records in the Electronic Medical Record Administrative data Linked Database (EMRALD) to assess the current state of primary care detection and management of chronic kidney disease in Ontario.ResultsSeventeen indicators made up the final list, with 1 under the category Prevalence, Incidence and Mortality; 4 under Screening, Diagnosis and Risk Factors; 11 under Management; and 1 under Referral to a Specialist. In a sample of 139 993 adult patients not on dialysis, 6848 (4.9%) had stage 3 or higher chronic kidney disease, with the average age of patients being 76.1 years (standard deviation [SD] 11.0); 62.9% of patients were female. Diagnosis and screening for chronic kidney disease were poorly performed. Only 27.1% of patients with stage 3 or higher disease had their diagnosis documented in their cumulative patient profile. Albumin-creatinine ratio testing was only performed for 16.3% of patients with a low estimated glomerular filtration rate (eGFR) and for 28.5% of patients with risk factors for chronic kidney disease. Family physicians performed relatively better with the management of chronic kidney disease, with 90.4% of patients with stage 3 or higher disease having an eGFR performed in the previous 18 months and 83.1% having a blood pressure recorded in the previous 9 months.InterpretationWe propose a set of measurable indicators to evaluate the quality of the management of chronic kidney disease in primary care. These indicators may be used to identify opportunities to improve current practice in Canada.

Dataset Information

Systematic approach to evaluating and confirming the utility of a suite of national health system performance (HSP) indicators in Canada: a modified Delphi study.

Objectives

Design

Setting

Participants

Primary and secondary outcome measures

Results

Conclusions

Publications

Systematic approach to evaluating and confirming the utility of a suite of national health system performance (HSP) indicators in Canada: a modified Delphi study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure