Missing Data Methods for Partial Correlations.
Ontology highlight
ABSTRACT:
SUBMITTER: D'Angelo GM
PROVIDER: S-EPMC3772686 | biostudies-literature | 2012 Dec
REPOSITORIES: biostudies-literature
Publications
Missing Data Methods for Partial Correlations.
D'Angelo Gina M GM Luo Jingqin J Xiong Chengjie C
Journal of biometrics & biostatistics 20121201 8
In the dementia area it is often of interest to study relationships among regional brain measures; however, it is often necessary to adjust for covariates. Partial correlations are frequently used to correlate two variables while adjusting for other variables. Complete case analysis is typically the analysis of choice for partial correlations with missing data. However, complete case analysis will lead to biased and inefficient results when the data are missing at random. We have extended the pa ...[more]
Similar Datasets
Project description:Secondary respondent data are underutilized because researchers avoid using these data in the presence of substantial missing data. We reviewed, critically evaluated, and tested potential solutions to this problem. Five strategies of dealing with missing partner data are reviewed: complete case analysis, inverse probability weighting, correction with a Heckman selection model, maximum likelihood estimation, and multiple imputation. Two approaches were used to evaluate the performance of these methods. First, we used data from the National Survey of Fertility Barriers (N = 1,666) to estimate a model predicting marital quality based on characteristics of women and their husbands. Second, we conducted a simulation based on these data testing the five methods and compared the results to estimates where the true value was known. We found that the maximum likelihood and multiple imputation methods were advantageous because they allow researchers to utilize all of the available information as well as produce less biased and more efficient estimates.
| S-EPMC4477957 | biostudies-literature
Project description:There has been a growing interest in multitask prediction in chemoinformatics, helped by the increasing use of deep neural networks in this field. This technique is applied to multitarget data sets, where compounds have been tested against different targets, with the aim of developing models to predict a profile of biological activities for a given compound. However, multitarget data sets tend to be sparse; i.e., not all compound-target combinations have experimental values. There has been little research on the effect of missing data on the performance of multitask methods. We have used two complete data sets to simulate sparseness by removing data from the training set. Different models to remove the data were compared. These sparse sets were used to train two different multitask methods, deep neural networks and Macau, which is a Bayesian probabilistic matrix factorization technique. Results from both methods were remarkably similar and showed that the performance decrease because of missing data is at first small before accelerating after large amounts of data are removed. This work provides a first approximation to assess how much data is required to produce good performance in multitask prediction exercises.
| S-EPMC5964064 | biostudies-literature
Project description:ObjectivesMissing laboratory data is a common issue, but the optimal method of imputation of missing values has not been determined. The aims of our study were to compare the accuracy of four imputation methods for missing completely at random laboratory data and to compare the effect of the imputed values on the accuracy of two clinical predictive models.DesignRetrospective cohort analysis of two large data sets.SettingA tertiary level care institution in Ann Arbor, Michigan.ParticipantsThe Cirrhosis cohort had 446 patients and the Inflammatory Bowel Disease cohort had 395 patients.MethodsNon-missing laboratory data were randomly removed with varying frequencies from two large data sets, and we then compared the ability of four methods-missForest, mean imputation, nearest neighbour imputation and multivariate imputation by chained equations (MICE)-to impute the simulated missing data. We characterised the accuracy of the imputation and the effect of the imputation on predictive ability in two large data sets.ResultsMissForest had the least imputation error for both continuous and categorical variables at each frequency of missingness, and it had the smallest prediction difference when models used imputed laboratory values. In both data sets, MICE had the second least imputation error and prediction difference, followed by the nearest neighbour and mean imputation.ConclusionsMissForest is a highly accurate method of imputation for missing laboratory data and outperforms other common imputation techniques in terms of imputation error and maintenance of predictive ability with imputed values in two clinical predicative models.
| S-EPMC3733317 | biostudies-literature
Project description:It is very common in instrumental variable studies for there to be missing instrument data. For example, in the Wisconsin Longitudinal Study one can use genotype data as a Mendelian randomization-style instrument, but this information is often missing when subjects do not contribute saliva samples, or when the genotyping platform output is ambiguous. Here we review missing-at-random assumptions one can use to identify instrumental variable causal effects, and discuss various approaches for estimation and inference. We consider likelihood-based methods, regression and weighting estimators, and doubly robust estimators. The likelihood-based methods yield the most precise inference, and are optimal under the model assumptions, while the doubly robust estimators can attain the nonparametric efficiency bound while allowing flexible nonparametric estimation of nuisance functions (e.g., instrument propensity scores). The regression and weighting estimators can sometimes be easiest to describe and implement. Our main contribution is an extensive review of this wide array of estimators under varied missing-at-random assumptions, along with discussion of asymptotic properties and inferential tools. We also implement many of the estimators in an analysis of the Wisconsin Longitudinal Study, to study effects of impaired cognitive functioning on depression.
| S-EPMC8025985 | biostudies-literature
Project description:Compound identification is a key component of data analysis in the applications of gas chromatography-mass spectrometry (GC-MS). Currently, the most widely used compound identification is mass spectrum matching, in which the dot product and its composite version are employed as spectral similarity measures. Several forms of transformations for fragment ion intensities have also been proposed to increase the accuracy of compound identification. In this study, we introduced partial and semipartial correlations as mass spectral similarity measures and applied them to identify compounds along with different transformations of peak intensity. The mixture versions of the proposed method were also developed to further improve the accuracy of compound identification. To demonstrate the performance of the proposed spectral similarity measures, the National Institute of Standards and Technology (NIST) mass spectral library and replicate spectral library were used as the reference library and the query spectra, respectively. Identification results showed that the mixture partial and semipartial correlations always outperform both the dot product and its composite measure. The mixture similarity with semipartial correlation has the highest accuracy of 84.6% in compound identification with a transformation of (0.53,1.3) for fragment ion intensity and m/z value, respectively.
| S-EPMC3418476 | biostudies-literature
Project description:The visibility and resolution of a tomographic reconstruction containing multiple copies of discrete particles can be enhanced by averaging subtomograms after they are corrected aligned. However, the "missing wedge" in electron tomography can easily lead to erroneous alignment. We have explored a Fourier space cross-correlation method with a proper weighting scheme to align and average different sets of volumetric data, each of which has different missing data due to the limited specimen tilts. This approach depends neither on a preexisting template, nor an exact knowledge of the geometry, orientation, or amount of the missing data. This paper introduces a procedure where the missing data might be gradually "filled in" by consecutively aligning and averaging volumes with different orientations of their missing data. We have validated these techniques by a set of simulated data with various symmetries and extent of missing data. We have also successfully applied these procedures to experimental cryo-electron tomographic data [Chang, J.T., Schmid, M.F., Rixon, F.J., and Chiu, W., 2007. Electron cryotomography reveals the portal in the herpesvirus capsid. J. Virol. 81, 2065-2068; Schmid, M.F., Paredes, A.M., Khant, H.A., Soyer, F., Aldrich, H.C., Chiu, W., and Shively, J.M., 2006. Structure of Halothiobacillus neapolitanus carboxysomes by cryo-electron tomography. J. Mol. Biol. 364, 526-535].
| S-EPMC2680136 | biostudies-literature
Project description:Inverse probability weights are increasingly used in epidemiological analysis, and estimation and application of weights to address a single bias are well discussed in the literature. Weights to address multiple biases simultaneously (i.e. a combination of weights) have almost exclusively been discussed related to marginal structural models in longitudinal settings where treatment weights (estimated first) are combined with censoring weights (estimated second). In this work, we examine two examples of combined weights for confounding and missingness in a time-fixed setting in which outcome or confounder data are missing, and the estimand is the marginal expectation of the outcome under a time-fixed treatment. We discuss the identification conditions, construction of combined weights and how assumptions of the missing data mechanisms affect this construction. We use a simulation to illustrate the estimation and application of the weights in the two examples. Notably, when only outcome data are missing, construction of combined weights is straightforward; however, when confounder data are missing, we show that in general we must follow a specific estimation procedure which entails first estimating missingness weights and then estimating treatment probabilities from data with missingness weights applied. However, if treatment and missingness are conditionally independent, then treatment probabilities can be estimated among the complete cases.
| S-EPMC9082798 | biostudies-literature
Project description:BACKGROUND:Multiple imputation (MI) is now widely used to handle missing data in longitudinal studies. Several MI techniques have been proposed to impute incomplete longitudinal covariates, including standard fully conditional specification (FCS-Standard) and joint multivariate normal imputation (JM-MVN), which treat repeated measurements as distinct variables, and various extensions based on generalized linear mixed models. Although these MI approaches have been implemented in various software packages, there has not been a comprehensive evaluation of the relative performance of these methods in the context of longitudinal data. METHOD:Using both empirical data and a simulation study based on data from the six waves of the Longitudinal Study of Australian Children (N?=?4661), we investigated the performance of a wide range of MI methods available in standard software packages for investigating the association between child body mass index (BMI) and quality of life using both a linear regression and a linear mixed-effects model. RESULTS:In this paper, we have identified and compared 12 different MI methods for imputing missing data in longitudinal studies. Analysis of simulated data under missing at random (MAR) mechanisms showed that the generally available MI methods provided less biased estimates with better coverage for the linear regression model and around half of these methods performed well for the estimation of regression parameters for a linear mixed model with random intercept. With the observed data, we observed an inverse association between child BMI and quality of life, with available data as well as multiple imputation. CONCLUSION:Both FCS-Standard and JM-MVN performed well for the estimation of regression parameters in both analysis models. More complex methods that explicitly reflect the longitudinal structure for these analysis models may only be needed in specific circumstances such as irregularly spaced data.
| S-EPMC6292063 | biostudies-literature
Project description:Multiple imputation (MI) is a well-established method for dealing with missing data. MI is computationally intensive when imputing missing covariates with high-dimensional outcome data (e.g., DNA methylation data in epigenome-wide association studies (EWAS)), because every outcome variable must be included in the imputation model to avoid biasing associations towards the null. Instead, EWAS analyses are reduced to only complete cases, limiting statistical power and potentially causing bias. We used simulations to compare 5 MI methods for high-dimensional data under 2 missingness mechanisms. All imputation methods had increased power over complete-case (C-C) analyses. Imputing missing values separately for each variable was computationally inefficient, but dividing sites at random into evenly sized bins improved efficiency and gave low bias. Methods imputing solely using subsets of sites identified by the C-C analysis suffered from bias towards the null. However, if these subsets were added into random bins of sites, this bias was reduced. The optimal methods were applied to an EWAS with missingness in covariates. All methods identified additional sites over the C-C analysis, and many of these sites had been replicated in other studies. These methods are also applicable to other high-dimensional data sets, including the rapidly expanding area of "-omics" studies.
| S-EPMC6825836 | biostudies-literature
Project description:PurposeRobust and reliable reconstruction of images from noisy and incomplete projection data holds significant potential for proliferation of cost-effective medical imaging technologies. Since conventional reconstruction techniques can generate severe artifacts in the recovered images, a notable line of research constitutes development of appropriate algorithms to compensate for missing data and to reduce noise. In the present work, we investigate the effectiveness of state-of-the-art methodologies developed for image inpainting and noise reduction to preserve the quality of reconstructed images from undersampled PET data. We aimed to assess and ascertain whether missing data recovery is best performed in the projection space prior to reconstruction or adjoined with the reconstruction step in image space.MethodsDifferent strategies for data recovery were investigated using realistic patient derived phantoms (brain and abdomen) in PET scanners with partial geometry (small and large gap structures). Specifically, gap filling strategies in projection space were compared with reconstruction based compensation in image space. The methods used for filling the gap structure in sinogram PET data include partial differential equation based techniques (PDE), total variation (TV) regularization, discrete cosine transform(DCT)-based penalized regression, and dictionary learning based inpainting (DLI). For compensation in image space, compressed sensing based image reconstruction methods were applied. These include the preconditioned alternating projection (PAPA) algorithm with first and higher order total variation (HOTV) regularization as well as dictionary learning based compressed sensing (DLCS). We additionally investigated the performance of the methods for recovery of missing data in the presence of simulated lesion. The impact of different noise levels in the undersampled sinograms on performance of the approaches were also evaluated.ResultsIn our first study (brain imaging), DLI was shown to outperform other methods for small gap structure in terms of root mean square error (RMSE) and structural similarity (SSIM), though having relatively high computational cost. For large gap structure, HOTV-PAPA produces better results. In the second study (abdomen imaging), again the best performance belonged to DLI for small gap, and HOTV-PAPA for large gap. In our experiments for lesion simulation on patient brain phantom data, the best performance in term of contrast recovery coefficient (CRC) for small gap simulation belonged to DLI, while in the case of large gap simulation, HOTV-PAPA outperformed others. Our evaluation of the impact of noise on performance of approaches indicated that in case of low and medium noise levels, DLI still produces favorable results among inpainting approaches. However, for high noise levels, the performance of PDE4 (variant of PDE) and DLI are very competitive.ConclusionsOur results showed that estimation of missing data in projection space as a preprocessing step before reconstruction can improve the quality of recovered images especially for small gap structures. However, when large portions of data are missing, compressed sensing techniques adjoined with the reconstruction step in image space were the best strategy.
| S-EPMC6329391 | biostudies-literature