Dataset Information

The Importance of Integrating Clinical Relevance and Statistical Significance in the Assessment of Quality of Care--Illustrated Using the Swedish Stroke Register.

ABSTRACT:

Background

When profiling hospital performance, quality inicators are commonly evaluated through hospital-specific adjusted means with confidence intervals. When identifying deviations from a norm, large hospitals can have statistically significant results even for clinically irrelevant deviations while important deviations in small hospitals can remain undiscovered. We have used data from the Swedish Stroke Register (Riksstroke) to illustrate the properties of a benchmarking method that integrates considerations of both clinical relevance and level of statistical significance.

Methods

The performance measure used was case-mix adjusted risk of death or dependency in activities of daily living within 3 months after stroke. A hospital was labeled as having outlying performance if its case-mix adjusted risk exceeded a benchmark value with a specified statistical confidence level. The benchmark was expressed relative to the population risk and should reflect the clinically relevant deviation that is to be detected. A simulation study based on Riksstroke patient data from 2008-2009 was performed to investigate the effect of the choice of the statistical confidence level and benchmark value on the diagnostic properties of the method.

Results

Simulations were based on 18,309 patients in 76 hospitals. The widely used setting, comparing 95% confidence intervals to the national average, resulted in low sensitivity (0.252) and high specificity (0.991). There were large variations in sensitivity and specificity for different requirements of statistical confidence. Lowering statistical confidence improved sensitivity with a relatively smaller loss of specificity. Variations due to different benchmark values were smaller, especially for sensitivity. This allows the choice of a clinically relevant benchmark to be driven by clinical factors without major concerns about sufficiently reliable evidence.

Conclusions

The study emphasizes the importance of combining clinical relevance and level of statistical confidence when profiling hospital performance. To guide the decision process a web-based tool that gives ROC-curves for different scenarios is provided.

SUBMITTER: Lindmark A

PROVIDER: S-EPMC4824466 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The Importance of Integrating Clinical Relevance and Statistical Significance in the Assessment of Quality of Care--Illustrated Using the Swedish Stroke Register.

Lindmark Anita A van Rompaye Bart B Goetghebeur Els E Glader Eva-Lotta EL Eriksson Marie M

PloS one 20160407 4

<h4>Background</h4>When profiling hospital performance, quality inicators are commonly evaluated through hospital-specific adjusted means with confidence intervals. When identifying deviations from a norm, large hospitals can have statistically significant results even for clinically irrelevant deviations while important deviations in small hospitals can remain undiscovered. We have used data from the Swedish Stroke Register (Riksstroke) to illustrate the properties of a benchmarking method that ...[more]

PMID: 27054326

Similar Datasets

Project description:BackgroundSequencing of the human genome and the subsequent analyses have produced immense volumes of data. The technological advances have opened new windows into genomics beyond the DNA sequence. In parallel, clinical practice generate large amounts of data. This represents an underused data source that has much greater potential in translational research than is currently realized. This research aims at implementing a translational medicine informatics platform to integrate clinical data (disease diagnosis, diseases activity and treatment) of Rheumatoid Arthritis (RA) patients from Karolinska University Hospital and their research database (biobanks, genotype variants and serology) at the Center for Molecular Medicine, Karolinska Institutet.MethodsRequirements engineering methods were utilized to identify user requirements. Unified Modeling Language and data modeling methods were used to model the universe of discourse and data sources. Oracle11g were used as the database management system, and the clinical development center (CDC) was used as the application interface. Patient data were anonymized, and we employed authorization and security methods to protect the system.ResultsWe developed a user requirement matrix, which provided a framework for evaluating three translation informatics systems. The implementation of the CDC successfully integrated biological research database (15172 DNA, serum and synovial samples, 1436 cell samples and 65 SNPs per patient) and clinical database (5652 clinical visit) for the cohort of 379 patients presents three profiles. Basic functionalities provided by the translational medicine platform are research data management, development of bioinformatics workflow and analysis, sub-cohort selection, and re-use of clinical data in research settings. Finally, the system allowed researchers to extract subsets of attributes from cohorts according to specific biological, clinical, or statistical features.ConclusionsResearch and clinical database integration is a real challenge and a road-block in translational research. Through this research we addressed the challenges and demonstrated the usefulness of CDC. We adhered to ethical regulations pertaining to patient data, and we determined that the existing software solutions cannot meet the translational research needs at hand. We used RA as a test case since we have ample data on active and longitudinal cohort.

Project description:ObjectivesBecause healthcare and community organisations and treatment methods are always changing, continuous changes might also be needed in questionnaires that register patient-reported outcomes (PRO) and patient-reported experiences (PRE) of healthcare interventions and community support. Thus, the aim of this study was to test the content and face validity, including the readability, of two questionnaires used by the Swedish national stroke register Riksstroke to register PRO and PRE at 3 and 12 months poststroke.DesignClinicians' and patients' knowledge and experiences of current care, rehabilitation, community support and functioning after stroke as well as comments noted regarding the content and layout of the questionnaires were retrieved in focus-groups with expert clinicians and in patient interviews analysed with content analysis. A workgroup of experts with experience in stroke care, rehabilitation and research repeatedly revised the questionnaires regarding content, layout and consistency throughout the validation process.ParticipantsThe participants included allied healthcare professionals, nurses and physicians with extensive experience of working with stroke care and rehabilitation (expert clinicians) as well as patients who had suffered a stroke 3 or 12 months earlier and who were purposefully selected among those who had completed and returned the 3-month questionnaire.SettingExpert clinicians met at their work place in focus-groups. Patients were interviewed where they resided, that is, in their home or nursing home, including rural, town and city areas in Sweden.ResultsBased on clinical expertise and comments from the patients (n=47), the questionnaires were revised and then found to be valid in terms of content validity and face validity, including readability.ConclusionsThe present evaluation emphasises the need for testing aspects of validity, including readability, of questionnaires addressing PRO and PRE and for the recurrent revision of such questionnaires in order to maintain their validity in a society undergoing constant change.

Dataset Information

The Importance of Integrating Clinical Relevance and Statistical Significance in the Assessment of Quality of Care--Illustrated Using the Swedish Stroke Register.

Background

Methods

Results

Conclusions

Publications

The Importance of Integrating Clinical Relevance and Statistical Significance in the Assessment of Quality of Care--Illustrated Using the Swedish Stroke Register.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure