Dataset Information

Modelling hospital outcome: problems with endogeneity.

ABSTRACT:

Background

Mortality modelling in the critical care paradigm traditionally uses logistic regression, despite the availability of estimators commonly used in alternate disciplines. Little attention has been paid to covariate endogeneity and the status of non-randomized treatment assignment. Using a large registry database, various binary outcome modelling strategies and methods to account for covariate endogeneity were explored.

Methods

Patient mortality data was sourced from the Australian & New Zealand Intensive Society Adult Patient Database for 2016. Hospital mortality was modelled using logistic, probit and linear probability (LPM) models with intensive care (ICU) providers as fixed (FE) and random (RE) effects. Model comparison entailed indices of discrimination and calibration, information criteria (AIC and BIC) and binned residual analysis. Suspect covariate and ventilation treatment assignment endogeneity was identified by correlation between predictor variable and hospital mortality error terms, using the Stata™ "eprobit" estimator. Marginal effects were used to demonstrate effect estimate differences between probit and "eprobit" models.

Results

The cohort comprised 92,693 patients from 124 intensive care units (ICU) in calendar year 2016. Patients mean age was 61.8 (SD 17.5) years, 41.6% were female and APACHE III severity of illness score 54.5(25.6); 43.7% were ventilated. Of the models considered in predicting hospital mortality, logistic regression (with or without ICU FE) and RE logistic regression dominated, more so the latter using information criteria indices. The LPM suffered from many predictions outside the unit [0,1] interval and both poor discrimination and calibration. Error terms of hospital length of stay, an independent risk of death score and ventilation status were correlated with the mortality error term. Marked differences in the ventilation mortality marginal effect was demonstrated between the probit and the "eprobit" models which were scenario dependent. Endogeneity was not demonstrated for the APACHE III score.

Conclusions

Logistic regression accounting for provider effects was the preferred estimator for hospital mortality modelling. Endogeneity of covariates and treatment variables may be identified using appropriate modelling, but failure to do so yields problematic effect estimates.

SUBMITTER: Moran JL

PROVIDER: S-EPMC8215743 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background: Early prognostication is a major challenge after out-of-hospital cardiac arrest (OHCA). Aims: We hypothesized that a genome-wide analysis of blood gene expression could offer new prognostic tools and lines of research. Methods: Sixty-nine patients were enrolled from an ancillary study of the clinical trial NCT00999583 that tested the effect of erythropoietin (EPO) after OHCA. Blood samples were collected in comatose survivors of OHCA at hospital admission and 1 and 3 days after resuscitation. Gene expression profiles were analyzed (Illumina HumanHT-12 V4 BeadChip; >34,000 genes). Patients were classified into two categories representing neurologically favorable outcome (Cerebral Performance Category [CPC] = 1-2) vs unfavorable outcome (CPC>2) at Day 60 after OHCA. Differential and functional enrichment analyses were performed to compare transcriptomic profiles between these two categories. Results: Among the 69 enrolled patients, 33 and 36 patients were treated or not by EPO, respectively. Among them, 42% had a favorable neurological outcome in both groups. EPO did not affect the transcriptomic response at Day-0 and 1 after OHCA. In contrast, 76 transcripts differed at Day-0 between patients with unfavorable vs favorable neurological outcome. This signature persisted at Day-1 after OHCA. Functional enrichment analysis revealed a down-regulation of adaptive immunity with concomitant up-regulation of innate immunity and inflammation in patients with unfavorable vs favorable neurological outcome. The transcription of many genes of the HLA family was decreased in patients with unfavorable vs favorable neurological outcome. Concomitantly, neutrophil activation and inflammation were observed. Up-stream regulators analysis showed the implication of numerous factors involved in cell cycle and damages. A logistic regression including a set of genes allowed a reliable prediction of the clinical outcomes (Specificity=88%; Hit Rate=83%). Conclusions: A transcriptomic signature involving a counterbalance between adaptive and innate immune responses is able to predict neurological outcome very early after hospital admission after OHCA. This deserves confirmation in a larger population.

Project description:BackgroundUsability testing of medical devices are mandatory for market access. The testings' goal is to identify usability problems that could cause harm to the user or limit the device's effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability testing is estimating the total number of problems, in order to assess the completeness of the discovery process. Today's margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems.MethodsThe matrix-based model was based on the full discovery matrix (including unobserved columns) and not solely on a summary of the data (e.g. the margins). This model also circumvents a drawback of margin-based methods by simultaneously estimating the model's parameters and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which reflects a real-life setting. As suggested in the usability literature, we assumed that the probability of detection had a logit-normal distribution.ResultsWe assessed the matrix-based method's performance in a range of settings reflecting real-life usability testing and with heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability) in a wide range of settings. We also applied our method to five real datasets from usability testing.ConclusionsEstimation models (and particularly matrix-based models) are of value in estimating and monitoring the detection process during usability testing. Matrix-based models have a solid mathematical grounding and, with a view to facilitating the decision-making process for both regulators and device manufacturers, should be incorporated into current standards.