Dataset Information

An empirical comparison of key statistical attributes among potential ICU quality indicators.

ABSTRACT: Good quality indicators should have face validity, relevance to patients, and be able to be measured reliably. Beyond these general requirements, good quality indicators should also have certain statistical properties, including sufficient variability to identify poor performers, relative insensitivity to severity adjustment, and the ability to capture what providers do rather than patients' characteristics. We assessed the performance of candidate indicators of ICU quality on these criteria. Indicators included ICU readmission, mortality, several length of stay outcomes, and the processes of venous-thromboembolism and stress ulcer prophylaxis provision.Retrospective cohort study.One hundred thirty-eight U.S. ICUs from 2001-2008 in the Project IMPACT database.Two hundred sixty-eight thousand eight hundred twenty-four patients discharged from U.S. ICUs.None.We assessed indicators' (1) variability across ICU-years; (2) degree of influence by patient vs. ICU and hospital characteristics using the Omega statistic; (3) sensitivity to severity adjustment by comparing the area under the receiver operating characteristic curve (AUC) between models including vs. excluding patient variables, and (4) correlation between risk adjusted quality indicators using a Spearman correlation. Large ranges of among-ICU variability were noted for all quality indicators, particularly for prolonged length of stay (4.7-71.3%) and the proportion of patients discharged home (30.6-82.0%), and ICU and hospital characteristics outweighed patient characteristics for stress ulcer prophylaxis (?, 0.43; 95% CI, 0.34-0.54), venous thromboembolism prophylaxis (?, 0.57; 95% CI, 0.53-0.61), and ICU readmissions (?, 0.69; 95% CI, 0.52-0.90). Mortality measures were the most sensitive to severity adjustment (area under the receiver operating characteristic curve % difference, 29.6%); process measures were the least sensitive (area under the receiver operating characteristic curve % differences: venous thromboembolism prophylaxis, 3.4%; stress ulcer prophylaxis, 2.1%). None of the 10 indicators was clearly and consistently correlated with a majority of the other nine indicators.No indicator performed optimally across assessments. Future research should seek to define and operationalize quality in a way that is relevant to both patients and providers.

SUBMITTER: Brown SE

PROVIDER: S-EPMC4212919 | biostudies-literature | 2014 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An empirical comparison of key statistical attributes among potential ICU quality indicators.

Brown Sydney E S SE Ratcliffe Sarah J SJ Halpern Scott D SD

Critical care medicine 20140801 8

<h4>Objective</h4>Good quality indicators should have face validity, relevance to patients, and be able to be measured reliably. Beyond these general requirements, good quality indicators should also have certain statistical properties, including sufficient variability to identify poor performers, relative insensitivity to severity adjustment, and the ability to capture what providers do rather than patients' characteristics. We assessed the performance of candidate indicators of ICU quality on ...[more]

PMID: 24717464

Similar Datasets

Project description:BackgroundExcess morbidity and mortality following critical illness is increasingly attributed to potentially avoidable complications occurring as a result of complex ICU management (Berenholtz et al., J Crit Care 17:1-2, 2002; De Vos et al., J Crit Care 22:267-74, 2007; Zimmerman J Crit Care 1:12-5, 2002). Routine measurement of quality indicators (QIs) through an Electronic Health Record (EHR) or registries are increasingly used to benchmark care and evaluate improvement interventions. However, existing indicators of quality for intensive care are derived almost exclusively from relatively narrow subsets of ICU patients from high-income healthcare systems. The aim of this scoping review is to systematically review the literature on QIs for evaluating critical care, identify QIs, map their definitions, evidence base, and describe the variances in measurement, and both the reported advantages and challenges of implementation.MethodWe searched MEDLINE, EMBASE, CINAHL, and the Cochrane libraries from the earliest available date through to January 2019. To increase the sensitivity of the search, grey literature and reference lists were reviewed. Minimum inclusion criteria were a description of one or more QIs designed to evaluate care for patients in ICU captured through a registry platform or EHR adapted for quality of care surveillance.ResultsThe search identified 4780 citations. Review of abstracts led to retrieval of 276 full-text articles, of which 123 articles were accepted. Fifty-one unique QIs in ICU were classified using the three components of health care quality proposed by the High Quality Health Systems (HQSS) framework. Adverse events including hospital acquired infections (13.7%), hospital processes (54.9%), and outcomes (31.4%) were the most common QIs identified. Patient reported outcome QIs accounted for less than 6%. Barriers to the implementation of QIs were described in 35.7% of articles and divided into operational barriers (51%) and acceptability barriers (49%).ConclusionsDespite the complexity and risk associated with ICU care, there are only a small number of operational indicators used. Future selection of QIs would benefit from a stakeholder-driven approach, whereby the values of patients and communities and the priorities for actionable improvement as perceived by healthcare providers are prioritized and include greater focus on measuring discriminable processes of care.

Project description:BackgroundMany definitions are being used to conceptualize child health problems. With survey data, commonly used indicators for identifying children with health problems have included chronic condition checklists, measures of activity limitations, elevated service use, and health utility thresholds. This study compares these different indicators in terms of the prevalence rates elicited, and in terms of how the subgroups identified differ.MethodsSecondary data analyses used data from the National Longitudinal Survey of Children and Youth, which surveyed a nationally representative sample of Canadian children (n = 13,790). Descriptive analyses compared healthy children to those with health problems, as classified by any of the key indicators. Additional analyses examined differences between subgroups of children captured by a single indicator and those described as having health problems by multiple indicators.ResultsThis study demonstrates that children captured by any of the indicators had poorer health than healthy children, despite the fact that over half the sample (52.2%) was characterized as having a health problem by at least one indicator. Rates of child ill health differed by indicator; 5.6% had an activity limitation, 9.2% exhibited a severe health difficulty, 31.7% reported a chronic condition, and 36.6% had elevated service use. Further, the four key indicators captured different types of children. Indicator groupings differed on child and socio-demographic factors. Compared to children identified by more than one indicator, those identified only by the severe health difficulty indicator displayed more cognitive problems (p < 0.0001), those identified only by the chronic condition checklist had a greater likelihood of reporting allergies or asthma (p < 0.0001), and those identified as having elevated service use only were more affluent (p = 0.01) and showed better overall health (p < 0.0001). Children identified by only a single indicator were less likely to have serious health problems than those identified by two or more indicators.ConclusionWe provide information useful to researchers when selecting indicators from survey data to identify children with health problems. Researchers and policy makers need to be aware of the impact of such definitions on prevalence rates as well as on the composition of children classified as being in poor health.

Project description:BackgroundThe Interrupted Time Series (ITS) is a robust design for evaluating public health and policy interventions or exposures when randomisation may be infeasible. Several statistical methods are available for the analysis and meta-analysis of ITS studies. We sought to empirically compare available methods when applied to real-world ITS data.MethodsWe sourced ITS data from published meta-analyses to create an online data repository. Each dataset was re-analysed using two ITS estimation methods. The level- and slope-change effect estimates (and standard errors) were calculated and combined using fixed-effect and four random-effects meta-analysis methods. We examined differences in meta-analytic level- and slope-change estimates, their 95% confidence intervals, p-values, and estimates of heterogeneity across the statistical methods.ResultsOf 40 eligible meta-analyses, data from 17 meta-analyses including 282 ITS studies were obtained (predominantly investigating the effects of public health interruptions (88%)) and analysed. We found that on average, the meta-analytic effect estimates, their standard errors and between-study variances were not sensitive to meta-analysis method choice, irrespective of the ITS analysis method. However, across ITS analysis methods, for any given meta-analysis, there could be small to moderate differences in meta-analytic effect estimates, and important differences in the meta-analytic standard errors. Furthermore, the confidence interval widths and p-values for the meta-analytic effect estimates varied depending on the choice of confidence interval method and ITS analysis method.ConclusionsOur empirical study showed that meta-analysis effect estimates, their standard errors, confidence interval widths and p-values can be affected by statistical method choice. These differences may importantly impact interpretations and conclusions of a meta-analysis and suggest that the statistical methods are not interchangeable in practice.

Project description:BackgroundA substantial degree of variability in practices exists amongst donor hospitals regarding the donor detection, determination of brain death, application of donor management techniques or achievement of donor management goals. A possible strategy to standardize the donation process and to optimize outcomes could lie in the implementation of a care pathway. The aim of the study was to identify and select a set of relevant key interventions and quality indicators in order to develop a specific care pathway for donation after brain death and to rigorously evaluate its impact.MethodsA RAND modified three-round Delphi approach was used to build consensus within a single country about potential key interventions and quality indicators identified in existing guidelines, review articles, process flow diagrams and the results of the Organ Donation European Quality System (ODEQUS) project. Comments and additional key interventions and quality indicators, identified in the first round, were evaluated in the following rounds and a subsequent physical meeting. The study was conducted over a 4-month time period in 2016.ResultsA multidisciplinary panel of 18 Belgian experts with different relevant backgrounds completed the three Delphi rounds. Out of a total of 80 key interventions assessed throughout the Delphi process, 65 were considered to contribute to the quality of care for the management of a potential donor after brain death; 11 out of 12 quality indicators were validated for relevance and feasibility. Detection of all potential donors after brain death in the intensive care unit and documentation of cause of no donation were rated as the most important quality indicators.ConclusionsUsing a RAND modified Delphi approach, consensus was reached for a set of 65 key interventions and 11 quality indicators for the management of a potential donor after brain death. This set is considered to be applicable in quality improvement programs for the care of potential donors after brain death, while taking into account each country's legislation and regulations regarding organ donation and transplantation.

Project description:The purpose of this study was to develop a method of performing routine periodical quality controls (QC) of CT systems by automatically analyzing key performance indicators (KPIs), obtainable from images of manufacturers' quality assurance (QA) phantoms. A KPI pertains to a measurable or determinable QC parameter that is influenced by other underlying fundamental QC parameters. The established KPIs are based on relationships between existing QC parameters used in the annual testing program of CT scanners at the Karolinska University Hospital in Stockholm, Sweden. The KPIs include positioning, image noise, uniformity, homogeneity, the CT number of water, and the CT number of air. An application (MonitorCT) was developed to automatically evaluate phantom images in terms of the established KPIs. The developed methodology has been used for two years in clinical routine, where CT technologists perform daily scans of the manufacturer's QA phantom and automatically send the images to MonitorCT for KPI evaluation. In the cases where results were out of tolerance, actions could be initiated in less than 10 min. 900 QC scans from two CT scanners have been collected and analyzed over the two-year period that MonitorCT has been active. Two types of errors have been registered in this period: a ring artifact was discovered with the image noise test, and a calibration error was detected multiple times with the CT number test. In both cases, results were outside the tolerances defined for MonitorCT, as well as by the vendor. Automated monitoring of KPIs is a powerful tool that can be used to supplement established QC methodologies. Medical physicists and other professionals concerned with the performance of a CT system will, using such methods, have access to comprehensive data on the current and historical (trend) status of the system such that swift actions can be taken in order to ensure the quality of the CT examinations, patient safety, and minimal disruption of service.

Dataset Information

An empirical comparison of key statistical attributes among potential ICU quality indicators.

Publications

An empirical comparison of key statistical attributes among potential ICU quality indicators.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets