Assessment of pharmacologic area under the curve when baselines are variable.
Ontology highlight
ABSTRACT:
SUBMITTER: Scheff JD
PROVIDER: S-EPMC3152796 | biostudies-literature | 2011 May
REPOSITORIES: biostudies-literature
Publications
Assessment of pharmacologic area under the curve when baselines are variable.
Scheff Jeremy D JD Almon Richard R RR Dubois Debra C DC Jusko William J WJ Androulakis Ioannis P IP
Pharmaceutical research 20110114 5
<h4>Purpose</h4>The area under the curve (AUC) is commonly used to assess the extent of exposure of a drug. The same concept can be applied to generally assess pharmacodynamic responses and the deviation of a signal from its baseline value. When the initial condition for the response of interest is not zero, there is uncertainty in the true value of the baseline measurement. This necessitates the consideration of the AUC relative to baseline to account for this inherent uncertainty and variabili ...[more]
Similar Datasets
Project description:BackgroundIncremental value (IncV) evaluates the performance change between an existing risk model and a new model. Different IncV metrics do not always agree with each other. For example, compared with a prescribed-dose model, an ovarian-dose model for predicting acute ovarian failure has a slightly lower area under the receiver operating characteristic curve (AUC) but increases the area under the precision-recall curve (AP) by 48%. This phenomenon of disagreement is not uncommon, and can create confusion when assessing whether the added information improves the model prediction accuracy.MethodsIn this article, we examine the analytical connections and differences between the AUC IncV (ΔAUC) and AP IncV (ΔAP). We also compare the true values of these two IncV metrics in a numerical study. Additionally, as both are semi-proper scoring rules, we compare them with a strictly proper scoring rule: the IncV of the scaled Brier score (ΔsBrS) in the numerical study.ResultsWe demonstrate that ΔAUC and ΔAP are both weighted averages of the changes (from the existing model to the new one) in separating the risk score distributions between events and non-events. However, ΔAP assigns heavier weights to the changes in higher-risk regions, whereas ΔAUC weights the changes equally. Due to this difference, the two IncV metrics can disagree, and the numerical study shows that their disagreement becomes more pronounced as the event rate decreases. In the numerical study, we also find that ΔAP has a wide range, from negative to positive, but the range of ΔAUC is much smaller. In addition, ΔAP and ΔsBrS are highly consistent, but ΔAUC is negatively correlated with ΔsBrS and ΔAP when the event rate is low.ConclusionsΔAUC treats the wins and losses of a new risk model equally across different risk regions. When neither the existing or new model is the true model, this equality could attenuate a superior performance of the new model for a sub-region. In contrast, ΔAP accentuates the change in the prediction accuracy for higher-risk regions.
| S-EPMC8278775 | biostudies-literature
Project description:The Area Under the Receiving Operating Characteristic Curve (AUC) is frequently used for assessing the overall accuracy of a diagnostic marker. However, estimation of AUC relies on knowledge of the true outcomes of subjects: diseased or non-diseased. Because disease verification based on a gold standard is often expensive and/or invasive, only a limited number of patients are sent to verification at doctors' discretion. Estimation of AUC is generally biased if only small verified samples are used and it is thus necessary to make corrections for such lack of information. However, correction based on the ignorable missingness assumption (or missing at random) is also biased if the missing mechanism indeed depends on the unknown disease outcome, which is called nonignorable missing. In this paper, we propose a propensity-score-adjustment method for estimating AUC based on the instrumental variable assumption when the missingness of disease status is nonignorable. The new method makes parametric assumptions on the verification probability, and the probability of being diseased for verified samples rather than for the whole sample. The proposed parametric assumption on the observed sample is easier to be verified than the parametric assumption on the full sample. We establish the asymptotic properties of the proposed estimators. A simulation study is performed to compare the proposed method with existing methods. The proposed method is also applied to an Alzheimer's disease data collected by National Alzheimer's Coordinating Center.
| S-EPMC6668995 | biostudies-literature
Project description:BackgroundRisk prediction models have been proposed for various diseases and are being improved as new predictors are identified. A major challenge is to determine whether the newly discovered predictors improve risk prediction. Decision curve analysis has been proposed as an alternative to the area under the curve and net reclassification index to evaluate the performance of prediction models in clinical scenarios. The decision curve computed using the net benefit can evaluate the predictive performance of risk models at a given or range of threshold probabilities. However, when the decision curves for 2 competing models cross in the range of interest, it is difficult to identify the best model as there is no readily available summary measure for evaluating the predictive performance. The key deterrent for using simple measures such as the area under the net benefit curve is the assumption that the threshold probabilities are uniformly distributed among patients.MethodsWe propose a novel measure for performing decision curve analysis. The approach estimates the distribution of threshold probabilities without the need of additional data. Using the estimated distribution of threshold probabilities, the weighted area under the net benefit curve serves as the summary measure to compare risk prediction models in a range of interest.ResultsWe compared 3 different approaches, the standard method, the area under the net benefit curve, and the weighted area under the net benefit curve. Type 1 error and power comparisons demonstrate that the weighted area under the net benefit curve has higher power compared to the other methods. Several simulation studies are presented to demonstrate the improvement in model comparison using the weighted area under the net benefit curve compared to the standard method.ConclusionsThe proposed measure improves decision curve analysis by using the weighted area under the curve and thereby improves the power of the decision curve analysis to compare risk prediction models in a clinical scenario.
| S-EPMC4949771 | biostudies-literature
Project description:Rationale: Interpretation of spirometry is influenced by inherent limitations and by the normal or predicted reference values used. For example, traditional spirometric parameters such as "distal" airflows do not provide sufficient differentiating capacity, especially for mixed patterns or small airway disease.Objectives: We assessed the utility of an alternative spirometric parameter (area under the expiratory flow-volume curve [AEX]) in differentiating between normal, obstruction, restriction, and mixed patterns, as well as in severity stratification of traditional functional impairments.Methods: We analyzed 15,308 spirometry tests in subjects who had same-day lung volume assessments in a pulmonary function laboratory. Using Global Lung Initiative predicted values and standard criteria for pulmonary function impairment, we assessed the diagnostic performance of AEX in best-split partition and artificial neural network models.Results: The average square root AEX values were 3.32, 1.81, 2.30, and 1.64 L?s-0.5 in normal, obstruction, restriction, and mixed patterns, respectively. As such, in combination with traditional spirometric measurements, the square root of AEX differentiated well between normal, obstruction, restriction, and mixed defects. Using forced expiratory volume in 1 second (FEV1), forced vital capacity (FVC), and FEV1/FVC z-scores plus the square root of AEX in a machine learning algorithm, diagnostic categorization of ventilatory impairments was accomplished with very low rates of misclassification (<9%). Especially for mixed ventilatory patterns, the neural network model performed best in improving the rates of diagnostic misclassification.Conclusions: Using a novel approach to lung function assessment in combination with traditional spirometric measurements, AEX differentiates well between normal, obstruction, restriction and mixed impairments, potentially obviating the need for more complex lung volume-based determinations.
| S-EPMC7193817 | biostudies-literature
Project description:ObjectivesTo assess differences in vancomycin AUC estimates from two common, clinically applied first-order pharmacokinetic equation methods compared with Bayesian estimates.MethodsA cohort of patients who received vancomycin and therapeutic drug monitoring was studied. First-order population pharmacokinetic equations were used to guide initial empirical dosing. After receipt of the first dose, patients had peak and trough serum levels drawn and steady-state AUC was estimated using first-order pharmacokinetic equations as standard care. We subsequently created a Bayesian model and used individual Empirical Bayes Estimates to precisely calculate vancomycin AUC24-48, AUC48-72 and AUC72-96 in this cohort. AUC at steady state (AUCSS) differences from the first-order methods were compared numerically and categorically (i.e. below, within or above 400-600 mg·h/L) to Bayesian AUCs, which served as the gold standard.ResultsA total of 65 adult inpatients with 409 plasma samples were included in this analysis. A two-compartment intravenous infusion model with first-order elimination fit the data well. The mean of Bayesian AUC24-48 was not significantly different from AUC estimates from the two first-order pharmacokinetic equation methods (P = 0.68); however, Bayesian AUC48-72 and Bayesian AUC72-96 were both significantly different when compared with both first-order pharmacokinetic equation methods (P < 0.01 for each). At the patient level, categorical classifications of AUC estimates from the two first-order pharmacokinetic equation methods differed from categorizations derived from the Bayesian calculations. Categorical agreement was ∼50% between first-order and Bayesian calculations, with declining categorical agreement observed with longer treatment courses. Differences in categorical agreement between calculation methods could potentially result in different dose recommendations for the patient.ConclusionsBayesian-calculated AUCs between 48-72 and 72-96 h intervals were significantly different from first-order pharmacokinetic method-estimated AUCs at steady state. The various calculation methods resulted in different categorical classification, which could potentially lead to erroneous dosing adjustments in approximately half of the patients.
| S-EPMC9890897 | biostudies-literature
Project description:BACKGROUND: The receiver operating characteristic (ROC) curve is a fundamental tool to assess the discriminant performance for not only a single marker but also a score function combining multiple markers. The area under the ROC curve (AUC) for a score function measures the intrinsic ability for the score function to discriminate between the controls and cases. Recently, the partial AUC (pAUC) has been paid more attention than the AUC, because a suitable range of the false positive rate can be focused according to various clinical situations. However, existing pAUC-based methods only handle a few markers and do not take nonlinear combination of markers into consideration. RESULTS: We have developed a new statistical method that focuses on the pAUC based on a boosting technique. The markers are combined componentially for maximizing the pAUC in the boosting algorithm using natural cubic splines or decision stumps (single-level decision trees), according to the values of markers (continuous or discrete). We show that the resulting score plots are useful for understanding how each marker is associated with the outcome variable. We compare the performance of the proposed boosting method with those of other existing methods, and demonstrate the utility using real data sets. As a result, we have much better discrimination performances in the sense of the pAUC in both simulation studies and real data analysis. CONCLUSIONS: The proposed method addresses how to combine the markers after a pAUC-based filtering procedure in high dimensional setting. Hence, it provides a consistent way of analyzing data based on the pAUC from maker selection to marker combination for discrimination problems. The method can capture not only linear but also nonlinear association between the outcome variable and the markers, about which the nonlinearity is known to be necessary in general for the maximization of the pAUC. The method also puts importance on the accuracy of classification performance as well as interpretability of the association, by offering simple and smooth resultant score plots for each marker.
| S-EPMC2898798 | biostudies-other
Project description:Genome-wide association studies in human populations have facilitated the creation of genomic profiles which combine the effects of many associated genetic variants to predict risk of disease. The area under the receiver operator characteristic (ROC) curve is a well established measure for determining the efficacy of tests in correctly classifying diseased and non-diseased individuals. We use quantitative genetics theory to provide insight into the genetic interpretation of the area under the ROC curve (AUC) when the test classifier is a predictor of genetic risk. Even when the proportion of genetic variance explained by the test is 100%, there is a maximum value for AUC that depends on the genetic epidemiology of the disease, i.e. either the sibling recurrence risk or heritability and disease prevalence. We derive an equation relating maximum AUC to heritability and disease prevalence. The expression can be reversed to calculate the proportion of genetic variance explained given AUC, disease prevalence, and heritability. We use published estimates of disease prevalence and sibling recurrence risk for 17 complex genetic diseases to calculate the proportion of genetic variance that a test must explain to achieve AUC = 0.75; this varied from 0.10 to 0.74. We provide a genetic interpretation of AUC for use with predictors of genetic risk based on genomic profiles. We provide a strategy to estimate proportion of genetic variance explained on the liability scale from estimates of AUC, disease prevalence, and heritability (or sibling recurrence risk) available as an online calculator.
| S-EPMC2829056 | biostudies-literature
Project description:BACKGROUND:This study analyzed the relationship between vancomycin area under the concentration-time curve (AUC) and acute kidney injury (AKI) reported across recent studies. METHODS:A systematic review of PubMed, Medline, Scopus, and compiled references was conducted. We included randomized cohort and case-control studies that reported vancomycin AUCs and risk of AKI (from 1990 to 2018). The primary outcome was AKI, defined as an increase in serum creatinine of ≥0.5 mg/L or a 50% increase from baseline on ≥2 consecutive measurements. Odds ratios (ORs) with 95% confidence intervals (CIs) were calculated. Primary analyses compared the impact of AUC cutpoint (greater than ~650 mg × hour/L) and AKI. Additional analysis compared AUC vs trough-guided monitoring on AKI incidence. RESULTS:Eight observational studies met inclusion/exclusion criteria with data for 2491 patients. Five studies reported first-24-hour AUCs (AUC0-24) and AKI, 2 studies reported 24- to 48-hour AUCs (AUC24-48) and AKI, and 2 studies reported AKI associated with AUC- vs trough-guided monitoring. AUC less than approximately 650 mg × hour/L was associated with decreased AKI for AUC0-24 (OR, 0.36 [95% CI, .23-.56]) as well as AUC24-48 (OR, 0.45 [95% CI, .27-.75]). AKI associated with the AUC monitoring strategy was significantly lower than trough-guided monitoring (OR, 0.68 [95% CI, .46-.99]). CONCLUSIONS:AUCs measured in the first or second 24 hours and lower than approximately 650 mg × hour/L may result in a decreased risk of AKI. Vancomycin AUC monitoring strategy may result in less vancomycin-associated AKI. Additional investigations are warranted.
| S-EPMC6853683 | biostudies-literature
Project description:BACKGROUND: A biomarker is usually used as a diagnostic or assessment tool in medical research. Finding an ideal biomarker is not easy and combining multiple biomarkers provides a promising alternative. Moreover, some biomarkers based on the optimal linear combination do not have enough discriminatory power. As a result, the aim of this study was to find the significant biomarkers based on the optimal linear combination maximizing the pAUC for assessment of the biomarkers. METHODS: Under the binormality assumption we obtain the optimal linear combination of biomarkers maximizing the partial area under the receiver operating characteristic curve (pAUC). Related statistical tests are developed for assessment of a biomarker set and of an individual biomarker. Stepwise biomarker selections are introduced to identify those biomarkers of statistical significance. RESULTS: The results of simulation study and three real examples, Duchenne Muscular Dystrophy disease, heart disease, and breast tissue example are used to show that our methods are most suitable biomarker selection for the data sets of a moderate number of biomarkers. CONCLUSIONS: Our proposed biomarker selection approaches can be used to find the significant biomarkers based on hypothesis testing.
| S-EPMC3923449 | biostudies-literature
Project description:Area under expiratory flow-volume curve (AEX) has been proposed recently to be a useful spirometric tool for assessing ventilatory patterns and impairment severity. We derive here normative reference values for AEX, based on age, gender, race, height and weight, and by using artificial neural network (ANN) algorithms. We analyzed 3567 normal spirometry tests with available AEX values, performed on subjects from two countries (United States and Spain). Regular linear or optimized regression and ANN models were built using traditional predictors of lung function. The ANN-based models outperformed the de novo regression-based equations for AEXpredicted and AEX z scores using race, gender, age, height and weight as predictor factors. We compared these reference values with previously developed equations for AEX (by gender and race), and found that the ANN models led to the most accurate predictions. When we compared the performance of ANN-based models in derivation/training, internal validation/testing, and external validation random groups, we found that the models based on pooling samples from various geographic areas outperformed the other models (in both central tendency and dispersion of the residuals, ameliorating any cohort effects). In a geographically diverse cohort of subjects with normal spirometry, we computed by both regression and ANN models several predicted equations and z scores for AEX, an alternative measurement of respiratory function. We found that the dynamic nature of the ANN allows for continuous improvement of the predictive models’ performance, thus promising that the AEX could become an essential tool in assessing respiratory impairment.
| S-EPMC7538954 | biostudies-literature