Dataset Information

Statistically Controlling for Confounding Constructs Is Harder than You Think.

ABSTRACT: Social scientists often seek to demonstrate that a construct has incremental validity over and above other related constructs. However, these claims are typically supported by measurement-level models that fail to consider the effects of measurement (un)reliability. We use intuitive examples, Monte Carlo simulations, and a novel analytical framework to demonstrate that common strategies for establishing incremental construct validity using multiple regression analysis exhibit extremely high Type I error rates under parameter regimes common in many psychological domains. Counterintuitively, we find that error rates are highest--in some cases approaching 100%--when sample sizes are large and reliability is moderate. Our findings suggest that a potentially large proportion of incremental validity claims made in the literature are spurious. We present a web application (http://jakewestfall.org/ivy/) that readers can use to explore the statistical properties of these and other incremental validity arguments. We conclude by reviewing SEM-based statistical approaches that appropriately control the Type I error rate when attempting to establish incremental validity.

SUBMITTER: Westfall J

PROVIDER: S-EPMC4816570 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Statistically Controlling for Confounding Constructs Is Harder than You Think.

Westfall Jacob J Yarkoni Tal T

PloS one 20160331 3

Social scientists often seek to demonstrate that a construct has incremental validity over and above other related constructs. However, these claims are typically supported by measurement-level models that fail to consider the effects of measurement (un)reliability. We use intuitive examples, Monte Carlo simulations, and a novel analytical framework to demonstrate that common strategies for establishing incremental construct validity using multiple regression analysis exhibit extremely high Type ...[more]

PMID: 27031707

Similar Datasets

Project description:BackgroundControlling unobserved confounding still remains a great challenge in observational studies, and a series of strict assumptions of the existing methods usually may be violated in practice. Therefore, it is urgent to put forward a novel method.MethodsWe are interested in the causal effect of an exposure on the outcome, which is always confounded by unobserved confounding. We show that, the causal effect of an exposure on a continuous or categorical outcome is nonparametrically identified through only two independent or correlated available confounders satisfying a non-linear condition on the exposure. Asymptotic theory and variance estimators are developed for each case. We also discuss an extension for more than two binary confounders.ResultsThe simulations show better estimation performance by our approach in contrast to the traditional regression approach adjusting for observed confounders. A real application is separately applied to assess the effects of Body Mass Index (BMI) on Systolic Blood Pressure (SBP), Diastolic Blood Pressure (DBP), Fasting Blood Glucose (FBG), Triglyceride (TG), Total Cholesterol (TC), High Density Lipoprotein (HDL) and Low Density Lipoprotein (LDL) with individuals in Shandong Province, China. Our results suggest that SBP increased 1.60 (95% CI: 0.99-2.93) mmol/L with per 1- kg/m2 higher BMI and DBP increased 0.37 (95% CI: 0.03-0.76) mmol/L with per 1- kg/m2 higher BMI. Moreover, 1- kg/m2 increase in BMI was causally associated with a 1.61 (95% CI: 0.96-2.97) mmol/L increase in TC, a 1.66 (95% CI: 0.91-55.30) mmol/L increase in TG and a 2.01 (95% CI: 1.09-4.31) mmol/L increase in LDL. However, BMI was not causally associated with HDL with effect value - 0.20 (95% CI: - 1.71-1.44). And, the effect value of FBG per 1- kg/m2 higher BMI was 0.56 (95% CI: - 0.24-2.18).ConclusionsWe propose a novel method to control unobserved confounders through double binary confounders satisfying a non-linear condition on the exposure which is easy to access.

Project description:The metabolome is the best representation of cancer phenotypes. Gene expression can be considered a confounding covariate affecting metabolite levels. Data integration across metabolomics and genomics to establish the biological relevance of cancer metabolism is challenging. This study aimed to eliminate the confounding effect of metabolic gene expression to reflect actual metabolite levels in microsatellite instability (MSI) cancers. In this study, we propose a new strategy using covariate-adjusted tensor classification in high dimensions (CATCH) models to integrate metabolite and metabolic gene expression data to classify MSI and microsatellite stability (MSS) cancers. We used datasets from the Cancer Cell Line Encyclopedia (CCLE) phase II project and treated metabolomic data as tensor predictors and data on gene expression of metabolic enzymes as confounding covariates. The CATCH model performed well, with high accuracy (0.82), sensitivity (0.66), specificity (0.88), precision (0.65), and F1 score (0.65). Seven metabolite features adjusted for metabolic gene expression, namely, 3-phosphoglycerate, 6-phosphogluconate, cholesterol ester, lysophosphatidylethanolamine (LPE), phosphatidylcholine, reduced glutathione, and sarcosine, were found in MSI cancers. Only one metabolite, Hippurate, was present in MSS cancers. The gene expression of phosphofructokinase 1 (PFKP), which is involved in the glycolytic pathway, was related to 3-phosphoglycerate. ALDH4A1 and GPT2 were associated with sarcosine. LPE was associated with the expression of CHPT1, which is involved in lipid metabolism. The glycolysis, nucleotide, glutamate, and lipid metabolic pathways were enriched in MSI cancers. We propose an effective CATCH model for predicting MSI cancer status. By controlling the confounding effect of metabolic gene expression, we identified cancer metabolic biomarkers and therapeutic targets. In addition, we provided the possible biology and genetics of MSI cancer metabolism.

Project description:BackgroundIn 2016, orthopaedic surgeons received nearly USD 300 million from industry, with the top 10% of recipients making more than 95% of the total amount. The degree to which gender may be associated with industry compensation has not been well explored; however, this may be confounded by a number of variables, including academic productivity, experience, and other factors. We wished to explore the variability in payment distribution by gender after controlling for these factors.Questions/purposes(1) Do men or women academic orthopaedic surgeons receive more payments from industry? (2) To what degree do any observed differences between the genders persist, even after accounting for identifiable factors, including academic rank, scholarly productivity, regional location of university, subspecialty selection as identified by fellowships completed, and years since completion of residency?MethodsThis study was a cross-sectional retrospective analysis of surgeons practicing in orthopaedic surgery academic departments in the United States. Academic orthopaedic surgery departments were identified using the Fellowship and Residency Electronic Interactive Database. Publicly available data on gender, academic rank, scholarly productivity, regional location of university, fellowships completed, and years since residency graduation were collected from institutional websites. Industry funding data for 2016 were obtained from the Centers for Medicare & Medicaid Services Open Payments Database, and scholarly productivity data through 2017 were collected from Scopus. A total of 2939 academic orthopaedic surgeons, 2620 (89%) men and 319 (11%) women from 126 programs were identified. Men and women surgeons were different in most of the variables collected, and all except region of university were associated with differences in industry payments.ResultsThe median payment for men surgeons was greater than that for women (USD 1027 [interquartile range USD 125-USD 9616] versus USD 177 [IQR USD 47-USD 1486]; difference of medians, USD 850; p < 0.001]. After accounting for potentially confounding variables like faculty rank, years since residency, H-index and subspecialty choice, women faculty members still received only 29% of payments received by otherwise comparable men orthopaedists (beta coefficient for gender = 0.29 [95% CI 0.20 to 0.44; p < 0.001]).ConclusionsWomen academic orthopaedic surgeons received only 29% of the industry payments received by men, even after controlling for faculty rank, years since residency, H-index, and subspecialty selection. This gender-related disparity may hinder the career advancement of women orthopaedic surgeons.Clinical relevanceIncreased transparency by companies can help guide orthopaedic surgeons who wish to receive industry funding.

Dataset Information

Statistically Controlling for Confounding Constructs Is Harder than You Think.

Publications

Statistically Controlling for Confounding Constructs Is Harder than You Think.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets