Handling over-dispersion of performance indicators.
Ontology highlight
ABSTRACT: Objectives
A problem can arise when a performance indicator shows substantially more variability than would be expected by chance alone, since ignoring such "over-dispersion" could lead to a large number of institutions being inappropriately classified as "abnormal". A number of options for handling this phenomenon are investigated, ranging from improved risk stratification to fitting a statistical model that robustly estimates the degree of over-dispersion.Design
Retrospective analysis of publicly available data on survival following coronary artery bypass grafts, emergency readmission rates, and teenage pregnancies.Setting
NHS trusts in England.Results
Funnel plots clearly show the influence of the method chosen for dealing with over-dispersion on the "banding" a trust receives. Both multiplicative and additive approaches are feasible and give intuitively reasonable results, but the additive random effects formulation appears to have a stronger conceptual foundation.Conclusion
A random effects model may offer a reasonable solution. This method has now been adopted by the UK Healthcare Commission in their derivation of star ratings.
SUBMITTER: Spiegelhalter DJ
PROVIDER: S-EPMC1744077 | biostudies-literature | 2005 Oct
REPOSITORIES: biostudies-literature
Publications
Handling over-dispersion of performance indicators.
Quality & safety in health care 20051001 5
<h4>Objectives</h4>A problem can arise when a performance indicator shows substantially more variability than would be expected by chance alone, since ignoring such "over-dispersion" could lead to a large number of institutions being inappropriately classified as "abnormal". A number of options for handling this phenomenon are investigated, ranging from improved risk stratification to fitting a statistical model that robustly estimates the degree of over-dispersion.<h4>Design</h4>Retrospective a ...[more]
Similar Datasets
Project description:Reaction time (RT) and RT variability are core components of cognitive performance that can be captured through brief and easy-to-administer tasks of simple RT and choice RT. The current study aims to describe age-related differences in cognitive performance, toward better characterizing normative performance across the lifespan. We examined mean and variability of response times on a simple RT and choice RT tasks in a large and diverse web-based sample (10,060 visitors to TestMyBrain.org). We also examined lifespan-related differences in response time variability using multiple different approaches (raw variability, mean scaled variability, and mean residualized variability). These analyses revealed significant heterogeneity in the patterns of age-related differences in performance, across metrics and within different estimates of the same metric. Based on segmented regression analysis, age of peak performance differed significantly across metrics, with young adults having the best performance based on measures of median RT, middle age adults at peak on certain measures of RT variability (standard deviation and coefficient of variability), and older adults showing the best performance based on accuracy and mean-corrected RT variability. Our results indicate that no single measure of cognitive performance and performance variability produces the same findings with respect to age related change, with further work needed to establish the validity of particular metrics for different applications.
| S-EPMC7068851 | biostudies-literature
Project description:BackgroundHigh midlife and late-life adiposity may increase risk for dementia. Late-life decrease in body mass index (BMI) or body weight within several years of a dementia diagnosis has also been reported. Differences in study designs and analyses may provide different pictures of this relationship.MethodsThirty-two years of longitudinal body weight, BMI, waist circumference, and waist-to-hip ratio (WHR) data, from the Prospective Population Study of Women in Sweden, were related to dementia. A representative sample of 1,462 nondemented women was followed from 1968 at ages 38-60 years, and subsequently in 1974, 1980, 1992, and 2000, using neuropsychiatric, anthropometric, clinical, and other measurements. Cox proportional hazards regression models estimated incident dementia risk by baseline factors. Logistic regression models including measures at each examination were related to dementia among surviving participants 32 years later.ResultsWhile Cox models showed no association between baseline anthropometric factors and dementia risk, logistic models showed that a midlife WHR greater than 0.80 increased risk for dementia approximately twofold (odds ratio 2.22, 95% confidence interval 1.00-4.94, p = 0.049) among surviving participants. Evidence for reverse causality was observed for body weight, BMI, and waist circumference in years preceding dementia diagnosis.ConclusionsAmong survivors to age 70, high midlife waist-to-hip ratio may increase odds of dementia. Traditional Cox models do not evidence this relationship. Changing anthropometric parameters in years preceding dementia onset indicate the dynamic nature of this seemingly simple relationship. There are midlife and late-life implications for dementia prevention, and analytical considerations related to identifying risk factors for dementia.
| S-EPMC2777073 | biostudies-literature
Project description:Cancers of the upper gastro-intestinal tract, including esophagus (gullet), stomach and small bowel, are amongst the deadliest malignancies. The main reason for their high mortality rate is that they are usually diagnosed late when curative treatments are no longer effective. However, these types of cancer generally arise from well-described pre-cancerous diseases, such as Barrett’s esophagus and gastric intestinal metaplasia. This provides an opportunity for clinicians to detect these pre-cancerous conditions early and offer adequate cure or clinical monitoring before they progress to cancer. A camera test (gastroscopy) is the gold-standard test to detect pre-cancerous diseases in these organs.
There has been limited research to set the standards for performance of a gastroscopy, especially with regards to diagnosis of pre-cancerous conditions, which require knowledge and skills by the physician performing the test (endoscopist). Therefore, the hypothesis behind this study is that the aforementioned pre-cancerous diseases are understudied and often go undetected. This study aims to understand how often endoscopists should diagnose these pre-cancerous diseases on routine gastroscopy and help define the standards to measure performance. The investigators will assess the following rates: i. how often endoscopists diagnose these pre-cancerous lesions during endoscopy; ii. How often these conditions are diagnosed on biopsies taken according to a standardized protocol; iii. How often these condition should have been diagnosed by the endoscopists based on the review of pictures by expert endoscopists. The investigators will also compare the rates of correct diagnosis by endoscopists with different levels of experience and based on the times spent to complete the diagnostic test.
Investigating these aspects will enhance the understanding of the medical community with regards to the diagnosis of these pre-cancerous lesions and set endoscopy standards to improve their early detection and treatment before they progress to cancer. This will translate to improved cancer prevention and benefit for patients.
| 2374191 | ecrin-mdr-crc
Project description:Ectonucleoside triphosphate diphosphohydrolase-1 hydrolyzes extracellular ATP and ADP to AMP. Previously, we showed that CD39 is expressed at several sites within the kidney and thus may impact the availability of type 2 purinergic receptor (P2-R) ligands. Because P2-Rs appear to regulate urinary concentrating ability, we have evaluated renal water handling in transgenic mice (TG) globally overexpressing hCD39. Under basal conditions, TG mice exhibited significantly impaired urinary concentration and decreased protein abundance of AQP2 in the kidney compared with wild-type (WT) mice. Urinary excretion of total nitrates/nitrites was significantly higher in TG mice, but the excretion of AVP or PGE(2) was equivalent to control WT mice. There were no significant differences in electrolyte-free water clearance or fractional excretion of sodium. Under stable hydrated conditions (gelled diet feeding), the differences between the WT and TG mice were negated, but the decrease in urine osmolality persisted. When water deprived, TG mice failed to adequately concentrate urine and exhibited impaired AVP responses. However, the increases in urinary osmolalities in response to subacute dDAVP or chronic AVP treatment were similar in TG and WT mice. These observations suggest that TG mice have impaired urinary concentrating ability despite normal AVP levels. We also note impaired AVP release in response to water deprivation but that TG kidneys are responsive to exogenous dDAVP or AVP. We infer that heightened nucleotide scavenging by increased levels of CD39 altered the release of endogenous AVP in response to dehydration. We propose that ectonucleotidases and modulated purinergic signaling impact urinary concentration and indicate potential utility of targeted therapy for the treatment of water balance disorders.
| S-EPMC3433867 | biostudies-literature
Project description:Purpose:The aim of this study was to analyze tactical and technical behavior across different ages and genders in young, elite beach volleyball players. Methods:Forty teams from the Under-18, Under-20, Under-21, and Under-22 from semifinals and finals of the 2016 World Championships and the 2016 European Championship were analyzed. The sample was composed of 69 sets. The variables studied included: Rally time, set time, match time, serve efficacy (standing serve, floating serve, and jump serve), setting efficacy (forearm, overhand, other, and 2nd attack), attack efficacy, and block efficacy. Student's t and Mann-Whitney U-tests were used to analyze specific differences between categories. Results:Significant differences (p < 0.05) in the pass performance, forearm pass in men (from 78.2 to 49.1%), and overhand pass in men (from 12.2 to 40.45%). In addition, in women forearm pass (from 88.5 to 76.3%) and overhand pass in women (from 1.2 to 9.35%). No significant differences in the effectiveness of attack, rally time, serve, and block efficacy. Conclusion:Tactical considerations and gender-specific differences in technical variables may be important for training in young players.
| S-EPMC6920118 | biostudies-literature
Project description:ObjectiveTo develop key performance indicators that evaluate the effectiveness of a prescription medication system.MethodsA modified RAND/UCLA appropriateness method was used to develop key performance indicators (KPIs) for a prescription medication system. A broad list of potential KPIs was compiled. A multidisciplinary group composed of 21 experts rated the potential KPIs. A face-to-face meeting was held following the first rating exercise to discuss each potential KPI individually. The expert panel undertook a final rating of KPIs. The final set of KPIs were those indicators where at least 80 percent of experts rated the indicator highly i.e. rating of ? 7 on a scale from 1 to 9.Results292 KPIs were identified from the published literature. After removing duplicates and combining similar indicators 71 KPIs were included. The final ranking resulted in six indicators being ranked 7 or higher by 80% of the respondents and an additional seven indicators being ranked 7 or higher by ?70 but ?80% of respondents. The six selected indicators include four specific disease areas, measure structural and process aspects of health service delivery, and assessed three of the domains of healthcare quality: efficiency, effectiveness, and safety.ConclusionsThese indicators are recommended as a starting point to assess the current performance of prescription medication systems. Consideration should be given to developing indicators in additional disease areas as well as indicators that measure the domains of timeliness and patient-centeredness. Future work should focus on the feasibility of measuring these indicators.
| S-EPMC6333341 | biostudies-literature
Project description:Cancer screening guidelines recommend that women over 50 years regularly be screened for breast, cervical and colorectal cancers. Population-based screening programs use performance indicators to monitor uptake for each type of cancer screening, but integrated measures of adherence across multiple screenings are rarely reported. Integrated measures of adherence that combine the three cancers cannot be inferred from measures of screening uptake of each cancer alone; nevertheless, they can help discern the proportion of women who, having received one or two types of screening, may be more amenable to receiving one additional screen, compared to those who haven't had any screening and may experience barriers to access screening such as distance, language, and so on. The focus of our search was to identify indicators of participation in the three cancers, therefore our search strategy included synonyms of integrated screening, cervical, breast and colorectal cancer screening. Additionally, we limited our search to studies published between 2000 and 2015, written in English, and pertaining to females over 50 years of age. The following databases were searched: MEDLINE, EMBASE, EBM Reviews, PubMed, PubMed Central, CINAHL, and Nursing Reference Center, as well as grey literature resources. Of the 78 initially retrieved articles, only 7 reported summary measures of screening across the three cancers. Overall, adherence to cervical, breast and colorectal cancer screening ranged from around 8% to 43%. Our review confirms that reports of screening adherence across breast, cervical and colorectal cancers are rare. This is surprising, as integrated cancer screening measures can provide additional insight into the needs of the target population that can help craft strategies to improve adherence to all three screenings.
| S-EPMC4982666 | biostudies-literature
Project description:We characterize the ecological, economic, and community performance of 21 major tuna fisheries, accounting for at least 77% of global tuna production, using the Fishery Performance Indicators. Our analysis reveals that the biggest variations in performance among tuna fisheries are driven by the final markets that they target: international sashimi market tuna fisheries considerably outperform a comparison set of 62 non-tuna fisheries in the Fishery Performance Indicator database, international canned tuna market fisheries perform similarly to the comparison set, and tuna fisheries supplying local markets in coastal states considerably underperform the comparison set. Differences among regional fishery management organizations primarily reflect regional species composition and market access, despite stark variation in governance, management, and other enabling conditions. With a legacy of open access, tuna's harvest sector performance is similar across all fisheries, reflecting only a normal return on the capital and skill invested: industrial vessels slightly outperform semi-industrial and artisanal vessels. Differences emerge in the post-harvest sector however, as value chains able to preserve quality and transport fish to high value markets outperform others.
| S-EPMC6456575 | biostudies-literature
Project description:Our aims were to measure anthropometric and oxygen uptake ( V? O2) variables in the laboratory, to measure kinetic and stride characteristics during a trail running time trial, and then analyse the data for correlations with trail running performance. Runners (13 men, 4 women: mean age: 29 ± 5 years; stature: 179.5 ± 0.8 cm; body mass: 69.1 ± 7.4 kg) performed laboratory tests to determine V? O2 max, running economy (RE), and anthropometric characteristics. On a separate day they performed an outdoor trail running time trial (two 3.5 km laps, total climb: 486 m) while we collected kinetic and time data. Comparing lap 2 with lap 1 (19:40 ± 1:57 min vs. 21:08 ± 2:09 min, P < 0.001), runners lost most time on the uphill sections and least on technical downhills (-2.5 ± 9.1 s). Inter-individual performance varied most for the downhills (CV > 25%) and least on flat terrain (CV < 10%). Overall stride cycle and ground contact time (GCT) were shorter in downhill than uphill sections (0.64 ± 0.03 vs. 0.84 ± 0.09 s; 0.26 ± 0.03 vs. 0.46 ± 0.90 s, both P < 0.001). Force impulse was greatest on uphill (248 ± 46 vs. 175 ± 24 Ns, P < 0.001) and related to GCT (r = 0.904, P < 0.001). Peak force was greater during downhill than during uphill running (1106 ± 135 vs. 959 ± 104 N, P < 0.01). Performance was related to absolute and relative V? O2 max (P < 0.01), vertical uphill treadmill speed (P < 0.001) and fat percent (P < 0.01). Running uphill involved the greatest impulse per step due to longer GCT while downhill running generated the highest peak forces. V? O2 max, vertical running speed and fat percent are important predictors for trail running performance. Performance between runners varied the most on downhills throughout the course, while pacing resembled a reversed J pattern. Future studies should focus on longer competition distances to verify these findings and with application of measures of 3D kinematics.
| S-EPMC6503082 | biostudies-literature
Project description:Quality performance indicators (QPIs) are used to monitor the delivery of cancer care. Neuroendocrine tumours (NETs) are a family of individually uncommon cancers that derive from neuroendocrine cells or their precursors, and can occur in most organs. There are currently no QPIs available for NETs and their heterogeneity makes QPI development difficult. CommNETs is a collaboration between NET clinicians, researchers and advocates in Canada, Australia and New Zealand. We created QPIs for NETs using a three-step consensus process. First, a multidisciplinary team used the nominal group technique to create candidates (n = 133) which were then curated into appropriateness statements (62 statements, 44 sub-statements). A two-stage modified RAND/UCLA Delphi consensus process was conducted: an online survey rated the statement appropriateness then the top-ranked statements (n = 20) were assessed in a face-to-face meeting. Finally, 10 QPIs met consensus criteria; documentation of primary site, proliferative index, differentiation, tumour board review, use of a structured pathology report, presence of distant metastasis, 5- and 10-year disease-free and overall survival. These NET QPIs will be trialed as a method to monitor and improve care for people with NETs and to facilitate international comparison.
| S-EPMC6780732 | biostudies-literature