Evaluating prognostic accuracy of biomarkers in nested case-control studies.
Ontology highlight
ABSTRACT:
SUBMITTER: Cai T
PROVIDER: S-EPMC3276269 | biostudies-other | 2012 Jan
REPOSITORIES: biostudies-other
Publications
Evaluating prognostic accuracy of biomarkers in nested case-control studies.
Biostatistics (Oxford, England) 20110819 1
Nested case-control (NCC) design is used frequently in epidemiological studies as a cost-effective subcohort sampling strategy to conduct biomarker research. Sampling strategy, on the other hoand, creates challenges for data analysis because of outcome-dependent missingness in biomarker measurements. In this paper, we propose inverse probability weighted (IPW) methods for making inference about the prognostic accuracy of a novel biomarker for predicting future events with data from NCC studies. ...[more]
Similar Datasets
Project description:Accurate risk prediction models are needed to identify different risk groups for individualized prevention and treatment strategies. In the Nurses' Health Study, to examine the effects of several biomarkers and genetic markers on the risk of rheumatoid arthritis (RA), a three-phase nested case-control (NCC) design was conducted, in which two sequential NCC subcohorts were formed with one nested within the other, and one set of new markers measured on each of the subcohorts. One objective of the study is to evaluate clinical values of novel biomarkers in improving upon existing risk models because of potential cost associated with assaying biomarkers. In this paper, we develop robust statistical procedures for constructing risk prediction models for RA and estimating the incremental value (IncV) of new markers based on three-phase NCC studies. Our method also takes into account possible time-varying effects of biomarkers in risk modeling, which allows us to more robustly assess the biomarker utility and address the question of whether a marker is better suited for short-term or long-term risk prediction. The proposed procedures are shown to perform well in finite samples via simulation studies.
| S-EPMC4819437 | biostudies-literature
Project description:To develop more targeted intervention strategies, an important research goal is to identify markers predictive of clinical events. A crucial step toward this goal is to characterize the clinical performance of a marker for predicting different types of events. In this article, we present statistical methods for evaluating the performance of a prognostic marker in predicting multiple competing events. To capture the potential time-varying predictive performance of the marker and incorporate competing risks, we define time- and cause-specific accuracy summaries by stratifying cases based on causes of failure. Such definition would allow one to evaluate the predictive accuracy of a marker for each type of event and compare its predictiveness across event types. Extending the nonparametric crude cause-specific receiver operating characteristics curve estimators by Saha and Heagerty (2010), we develop inference procedures for a range of cause-specific accuracy summaries. To estimate the accuracy measures and assess how covariates may affect the accuracy of a marker under the competing risk setting, we consider two forms of semiparametric models through the cause-specific hazard framework. These approaches enable a flexible modeling of the relationships between the marker and failure times for each cause, while efficiently accommodating additional covariates. We investigate the asymptotic property of the proposed accuracy estimators and demonstrate the finite sample performance of these estimators through simulation studies. The proposed procedures are illustrated with data from a prostate cancer prognostic study.
| S-EPMC3694786 | biostudies-literature
Project description:Biomarkers are playing an increasingly important role in disease screening, early detection, and risk prediction. The two-phase case-control sampling study design is widely used for the evaluation of candidate biomarkers. The sampling probabilities for cases and controls in the second phase can often depend on other covariates (sampling strata). This biased sampling can lead to invalid inference on a biomarker's classification accuracy if not properly accounted for. In this paper, we adopt the idea of inverse probability weighting and develop inverse probability weighting-based estimators for various measures of a biomarker's classification performance, including the points on the receiver operating characteristics (ROCs) curve, the area under the ROC curve (area under the curve), and the partial area under the curve. In particular, we consider classification accuracy estimators using sampling weights estimated conditionally on sampling strata and further improve their efficiency through the use of estimated weights that additionally take into account the auxiliary variables available from the phase-one cohort. We develop asymptotic properties of the proposed estimators and provide analytical variance for making inference. Extensive simulation studies demonstrate excellent performance of the proposed weighted estimators, while the traditional empirical estimator can be severely biased. We also investigate the advantages in efficiency gain for estimating various classification accuracy estimators through the use of auxiliary variables in addition to sampling strata and apply the proposed method to examples from a renal artery stenosis study and a prostate cancer study.
| S-EPMC6317859 | biostudies-literature
Project description:ObjectiveTo examine the association between interferon-β (IFN-β) and potential adverse events using population-based health administrative data in British Columbia, Canada.MethodsPatients with relapsing-remitting multiple sclerosis (RRMS) who were registered at a British Columbia Multiple Sclerosis Clinic (1995-2004) were eligible for inclusion and were followed up until death, absence from British Columbia, exposure to a non-IFN-β disease-modifying drug, or December 31, 2008. Incidence rates were estimated for each potential adverse event (selected a priori and defined with ICD-9/10 diagnosis codes from physician and hospital claims). A nested case-control study was conducted to assess the odds of previous IFN-β exposure for each potential adverse event with at least 30 cases. Cases were matched by age (±5 years), sex, and year of cohort entry, with up to 20 randomly selected (by incidence density sampling) controls. Odds ratios (ORs) with 95% confidence intervals (95% CIs) were estimated with conditional logistic regression adjusted for age at cohort entry.ResultsOf the 2,485 eligible patients, 77.9% were women, and 1,031 were treated with IFN-β during follow-up. From the incidence analyses, 27 of the 47 potential adverse events had at least 30 cases. Patients with incident stroke (ORadj 1.83, 95% CI 1.16-2.89), migraine (ORadj 1.55, 95% CI 1.18-2.04), depression (ORadj 1.33, 95% CI 1.13-1.56), and hematologic abnormalities (ORadj 1.32, 95% CI 1.01-1.72) were more likely to have previous exposure to IFN-β than controls.ConclusionsAmong patients with RRMS, IFN-β was associated with a 1.8- and 1.6-fold increase in the risk of stroke and migraine and 1.3-fold increases in depression and hematologic abnormalities.
| S-EPMC5567323 | biostudies-literature
Project description:The nested case-control (NCC) design have been widely adopted as a cost-effective solution in many large cohort studies for risk assessment with expensive markers, such as the emerging biologic and genetic markers. To analyze data from NCC studies, conditional logistic regression (Goldstein and Langholz, 1992; Borgan et al., 1995) and maximum likelihood (Scheike and Juul, 2004; Zeng et al., 2006) based methods have been proposed. However, most of these methods either cannot be easily extended beyond the Cox model (Cox, 1972) or require additional modeling assumptions. More generally applicable approaches based on inverse probability weighting (IPW) have been proposed as useful alternatives (Samuelsen, 1997; Chen, 2001; Samuelsen et al., 2007). However, due to the complex correlation structure induced by repeated finite risk set sampling, interval estimation for such IPW estimators remain challenging especially when the estimation involves non-smooth objective functions or when making simultaneous inferences about functions. Standard resampling procedures such as the bootstrap cannot accommodate the correlation and thus are not directly applicable. In this paper, we propose a resampling procedure that can provide valid estimates for the distribution of a broad class of IPW estimators. Simulation results suggest that the proposed procedures perform well in settings when analytical variance estimator is infeasible to derive or gives less optimal performance. The new procedures are illustrated with data from the Framingham Offspring Study to characterize individual level cardiovascular risks over time based on the Framingham risk score, C-reactive protein (CRP) and a genetic risk score.
| S-EPMC3891801 | biostudies-literature
Project description:The nested case-control and case-cohort designs are two main approaches for carrying out a substudy within a prospective cohort. This article adapts multiple imputation (MI) methods for handling missing covariates in full-cohort studies for nested case-control and case-cohort studies. We consider data missing by design and data missing by chance. MI analyses that make use of full-cohort data and MI analyses based on substudy data only are described, alongside an intermediate approach in which the imputation uses full-cohort data but the analysis uses only the substudy. We describe adaptations to two imputation methods: the approximate method (MI-approx) of White and Royston (2009) and the "substantive model compatible" (MI-SMC) method of Bartlett et al. (2015). We also apply the "MI matched set" approach of Seaman and Keogh (2015) to nested case-control studies, which does not require any full-cohort information. The methods are investigated using simulation studies and all perform well when their assumptions hold. Substantial gains in efficiency can be made by imputing data missing by design using the full-cohort approach or by imputing data missing by chance in analyses using the substudy only. The intermediate approach brings greater gains in efficiency relative to the substudy approach and is more robust to imputation model misspecification than the full-cohort approach. The methods are illustrated using the ARIC Study cohort. Supplementary Materials provide R and Stata code.
| S-EPMC6481559 | biostudies-literature
Project description:RationaleCross-sectional studies demonstrate an association between metabolic syndrome and impaired lung function.ObjectivesTo define if metabolic syndrome biomarkers are risk factors for loss of lung function after irritant exposure.MethodsA nested case-control study of Fire Department of New York personnel with normal pre-September 11th FEV(1) and who presented for subspecialty pulmonary evaluation before March 10, 2008. We correlated metabolic syndrome biomarkers obtained within 6 months of World Trade Center dust exposure with subsequent FEV(1). FEV(1) at subspecialty pulmonary evaluation within 6.5 years defined disease status; cases had FEV(1) less than lower limit of normal, whereas control subjects had FEV(1) greater than or equal to lower limit of normal.Measurements and main resultsClinical data and serum sampled at the first monitoring examination within 6 months of September 11, 2001, assessed body mass index, heart rate, serum glucose, triglycerides and high-density lipoprotein (HDL), leptin, pancreatic polypeptide, and amylin. Cases and control subjects had significant differences in HDL less than 40 mg/dl with triglycerides greater than or equal to 150 mg/dl, heart rate greater than or equal to 66 bpm, and leptin greater than or equal to 10,300 pg/ml. Each increased the odds of abnormal FEV(1) at pulmonary evaluation by more than twofold, whereas amylin greater than or equal to 116 pg/ml decreased the odds by 84%, in a multibiomarker model adjusting for age, race, body mass index, and World Trade Center arrival time. This model had a sensitivity of 41%, a specificity of 86%, and a receiver operating characteristic area under the curve of 0.77.ConclusionsAbnormal triglycerides and HDL and elevated heart rate and leptin are independent risk factors of greater susceptibility to lung function impairment after September 11, 2001, whereas elevated amylin is protective. Metabolic biomarkers are predictors of lung disease, and may be useful for assessing risk of impaired lung function in response to particulate inhalation.
| S-EPMC3297095 | biostudies-literature
Project description:BackgroundWe hypothesised that host-response biomarkers of viral infections might contribute to early identification of individuals infected with SARS-CoV-2, which is critical to breaking the chains of transmission. We aimed to evaluate the diagnostic accuracy of existing candidate whole-blood transcriptomic signatures for viral infection to predict positivity of nasopharyngeal SARS-CoV-2 PCR testing.MethodsWe did a nested case-control diagnostic accuracy study among a prospective cohort of health-care workers (aged ≥18 years) at St Bartholomew's Hospital (London, UK) undergoing weekly blood and nasopharyngeal swab sampling for whole-blood RNA sequencing and SARS-CoV-2 PCR testing, when fit to attend work. We identified candidate blood transcriptomic signatures for viral infection through a systematic literature search. We searched MEDLINE for articles published between database inception and Oct 12, 2020, using comprehensive MeSH and keyword terms for "viral infection", "transcriptome", "biomarker", and "blood". We reconstructed signature scores in blood RNA sequencing data and evaluated their diagnostic accuracy for contemporaneous SARS-CoV-2 infection, compared with the gold standard of SARS-CoV-2 PCR testing, by quantifying the area under the receiver operating characteristic curve (AUROC), sensitivities, and specificities at a standardised Z score of at least 2 based on the distribution of signature scores in test-negative controls. We used pairwise DeLong tests compared with the most discriminating signature to identify the subset of best performing biomarkers. We evaluated associations between signature expression, viral load (using PCR cycle thresholds), and symptom status visually and using Spearman rank correlation. The primary outcome was the AUROC for discriminating between samples from participants who tested negative throughout the study (test-negative controls) and samples from participants with PCR-confirmed SARS-CoV-2 infection (test-positive participants) during their first week of PCR positivity.FindingsWe identified 20 candidate blood transcriptomic signatures of viral infection from 18 studies and evaluated their accuracy among 169 blood RNA samples from 96 participants over 24 weeks. Participants were recruited between March 23 and March 31, 2020. 114 samples were from 41 participants with SARS-CoV-2 infection, and 55 samples were from 55 test-negative controls. The median age of participants was 36 years (IQR 27-47) and 69 (72%) of 96 were women. Signatures had little overlap of component genes, but were mostly correlated as components of type I interferon responses. A single blood transcript for IFI27 provided the highest accuracy for discriminating between test-negative controls and test-positive individuals at the time of their first positive SARS-CoV-2 PCR result, with AUROC of 0·95 (95% CI 0·91-0·99), sensitivity 0·84 (0·70-0·93), and specificity 0·95 (0·85-0·98) at a predefined threshold (Z score >2). The transcript performed equally well in individuals with and without symptoms. Three other candidate signatures (including two to 48 transcripts) had statistically equivalent discrimination to IFI27 (AUROCs 0·91-0·95).InterpretationOur findings support further urgent evaluation and development of blood IFI27 transcripts as a biomarker for early phase SARS-CoV-2 infection for screening individuals at high risk of infection, such as contacts of index cases, to facilitate early case isolation and early use of antiviral treatments as they emerge.FundingBarts Charity, Wellcome Trust, and National Institute of Health Research.
| S-EPMC8260104 | biostudies-literature
Project description:The nested case-control (NCC) design is a cost-effective sampling method to study the relationship between a disease and its risk factors in epidemiologic studies. NCC data are commonly analyzed using Thomas' partial likelihood approach under Cox's proportional hazards model with constant covariate effects. Here, we are interested in studying the potential time-varying effects of covariates in NCC studies and propose an estimation approach based on a kernel-weighted Thomas' partial likelihood. We establish asymptotic properties of the proposed estimator, propose a numerical approach to construct simultaneous confidence bands for time-varying coefficients, and develop a hypothesis testing procedure to detect time-varying coefficients. The proposed inference procedure is evaluated in simulations and applied to an NCC study of breast cancer in the New York University Women's Health Study.
| S-EPMC3294270 | biostudies-literature
Project description:Pooling biomarker data across multiple studies allows for examination of a wider exposure range than generally possible in individual studies, evaluation of population subgroups and disease subtypes with more statistical power, and more precise estimation of biomarker-disease associations. However, circulating biomarker measurements often require calibration to a single reference assay prior to pooling due to assay and laboratory variability across studies. We propose several methods for calibrating and combining biomarker data from nested case-control studies when reference assay data are obtained from a subset of controls in each contributing study. Specifically, we describe a two-stage calibration method and two aggregated calibration methods, named the internalized and full calibration methods, to evaluate the main effect of the biomarker exposure on disease risk and whether that association is modified by a potential covariate. The internalized method uses the reference laboratory measurement in the analysis when available and otherwise uses the estimated value derived from calibration models. The full calibration method uses calibrated biomarker measurements for all subjects, including those with reference laboratory measurements. Under the two-stage method, investigators complete study-specific analyses in the first stage followed by meta-analysis in the second stage. Our results demonstrate that the full calibration method is the preferred aggregated approach to minimize bias in point estimates. We also observe that the two-stage and full calibration methods provide similar effect and variance estimates but that their variance estimates are slightly larger than those from the internalized approach. As an illustrative example, we apply the three methods in a pooling project of nested case-control studies to evaluate (i) the association between circulating vitamin D levels and risk of stroke and (ii) how body mass index modifies the association between circulating vitamin D levels and risk of cardiovascular disease.
| S-EPMC8286552 | biostudies-literature