P value variability and subgroup testing.
Ontology highlight
ABSTRACT:
SUBMITTER: Horgan G
PROVIDER: S-EPMC8572204 | biostudies-literature | 2021 Dec
REPOSITORIES: biostudies-literature
Publications
This article discusses the variability and randomness of p values, the most widely used currency of evidence in nutritional and health studies. One implication of this, the importance of always testing interaction terms when subgroups are examined and presented separately is also discussed. ...[more]
Similar Datasets
Project description:BackgroundThe American Society of Anesthesiologists (ASA) Choosing Wisely Top-5 list of activities to avoid includes "Don't obtain baseline laboratory studies in patients without significant systemic disease (ASA I or II) undergoing low-risk surgery - specifically complete blood count, basic or comprehensive metabolic panel, coagulation studies when blood loss (or fluid shifts) is/are expected to be minimal." Accordingly, we define low-value preoperative tests (LVTs) as those performed before minor surgery in patients without significant systemic disease. The objective of the current study was to examine the extent, variability, drivers, and costs of LVTs before carpal tunnel release (CTR) surgeries in the US Veterans Health Administration (VHA).MethodsUsing fiscal year (FY) 2015-2017 data derived from the VHA Corporate Data Warehouse (CDW), we determined the overall national and facility-level rates and associated costs of receiving any of 8 common LVTs in the 30 days before CTR in ASA physical status (PS) I-II patients. We also examined the patient, procedure, and facility factors associated with receiving ≥1 LVT with mixed-effects logistic regression and the number of tests received with mixed-effects negative binomial regression.ResultsFrom FY15-17, 10,000 ASA class I-II patients received a CTR by 699 surgeons in 125 VHA facilities. Overall, 47.0% of patients had a CTR that was preceded by ≥1 LVT, with substantial variability between facilities (range = 0%-100%; interquartile range = 36.3%), representing $339,717 in costs. Older age and female sex were associated with higher odds of receiving ≥1 LVT. Local versus other modes of anesthesia were associated with lower odds of receiving ≥1 LVT. Several facilities experienced large (>25%) increases or decreases from FY15 to FY17 in the proportion of patients receiving ≥1 LVT.ConclusionsCounter to guidance from the ASA, we found that almost half of CTRs performed on ASA class I-II VHA patients were preceded by ≥1 LVT. Although the total cost of these tests is relatively modest, CTR is just one of many low-risk procedures (eg, trigger finger release, cataract surgery) that may involve similar preoperative testing practices. These results will inform site selection for qualitative investigation of the drivers of low-value testing and the development of interventions to improve preoperative testing practice, especially in locations where rates of LVT are high.
| S-EPMC6760302 | biostudies-literature
Project description:ImportanceThe Choosing Wisely guidelines indicate that preoperative testing is often unnecessary and wasteful for patients undergoing cataract operations. However, little is known about the impact of these widely disseminated guidelines within the US Veterans Health Administration (VHA) system.ObjectiveTo examine the extent, variability, associated factors, and costs of low-value tests (LVTs) prior to cataract operations in the VHA.Design, setting, and participantsThis cohort study examined records of all patients receiving cataract operations within the VHA in fiscal year 2017 (October 1, 2016, to September 31, 2017). Records from 135 facilities nationwide supporting both ambulatory and inpatient surgery were included.ExposuresA laboratory test occurring within 30 days prior to cataract surgery and within 30 days after clinic evaluation.Main outcomes and measuresOverall national and facility-level rates and associated costs of receiving any of 8 common LVTs in the 30 days prior to cataract surgery. The patient characteristics, procedure type, and facility-level factors associated with receiving at least 1 test, the number of tests received, and receipt of a bundle of 4 tests (complete blood count, basic metabolic profile, chest radiograph, and electrocardiogram).ResultsA total of 69 070 cataract procedures were identified among 50 106 patients (66 282 [96.0%] men; mean [SD] age, 71.7 [8.1] years; 53 837 [77.9%] White, 10 292 [14.9%] Black). Most of the patient population had either overweight (23 292 [33.7%] patients) or obesity (27 799 [40.2%] patients). Approximately 49% of surgical procedures (33 424 procedures) were preceded by 1 or more LVT with an overall LVT cost of $2 597 623. Among patients receiving LVTs, electrocardiography (7434 patients [29.9%]) was the most common, with some patients also receiving more costly tests, including chest radiographs (489 patients [8.2%]) and pulmonary function tests (127 patients [3.4%]). For receipt of any LVT, the intraclass correlation coefficient was 0.61 (P < .001) at the facility level and 0.06 (P < .001) at the surgeon level, indicating the substantial contribution of the facility to amount of tests given.Conclusions and relevanceDespite existing guidelines, use of LVTs prior to cataract surgery is both common and costly within a large, national integrated health care system. Our results suggest that publishing evidence-based guidelines alone-such as the Choosing Wisely campaign-may not sufficiently influence individual physician behavior, and that system-level efforts to directly deimplement LVTs may therefore necessary to effect sustained change.
| S-EPMC8103225 | biostudies-literature
Project description:There is a growing interest in clinical trials that investigate how patients may respond differently to an experimental treatment depending on the basis of some biomarker measured on a continuous scale, and in particular to identify some threshold value for the biomarker above which a positive treatment effect can be considered to have been demonstrated. This can be statistically challenging when the same data are used both to select the threshold and to test the treatment effect in the subpopulation that it defines. This paper describes a hierarchical testing framework to give familywise type I error rate control in this setting and proposes two specific tests that can be used within this framework. One, a simple test based on the estimated value from a linear regression model with treatment by biomarker interaction, is powerful but can lead to type I error rate inflation if the assumptions of the linear model are not met. The other is more robust to these assumptions, but can be slightly less powerful when the assumptions hold.
| S-EPMC11577705 | biostudies-literature
Project description:In many clinical applications, understanding when measurement of new markers is necessary to provide added accuracy to existing prediction tools could lead to more cost effective disease management. Many statistical tools for evaluating the incremental value (IncV) of the novel markers over the routine clinical risk factors have been developed in recent years. However, most existing literature focuses primarily on global assessment. Since the IncVs of new markers often vary across subgroups, it would be of great interest to identify subgroups for which the new markers are most/least useful in improving risk prediction. In this paper we provide novel statistical procedures for systematically identifying potential traditional-marker based subgroups in whom it might be beneficial to apply a new model with measurements of both the novel and traditional markers. We consider various conditional time-dependent accuracy parameters for censored failure time outcome to assess the subgroup-specific IncVs. We provide non-parametric kernel-based estimation procedures to calculate the proposed parameters. Simultaneous interval estimation procedures are provided to account for sampling variation and adjust for multiple testing. Simulation studies suggest that our proposed procedures work well in finite samples. The proposed procedures are applied to the Framingham Offspring Study to examine the added value of an inflammation marker, C-reactive protein, on top of the traditional Framingham risk score for predicting 10-year risk of cardiovascular disease.
| S-EPMC3633735 | biostudies-literature
Project description:BackgroundCardiopulmonary exercise testing is an increasingly common test and is considered the accepted standard for assessing exercise capacity. Quantifying variability is important to assess the instrument for quality control purposes. Though guidelines recommend biologic control testing, there are minimal data on how to do it. We sought to describe variability for oxygen consumption (V̇O2 ), carbon dioxide production (V̇CO2 ), and minute ventilation (V̇E) at various work rates under steady-state conditions in multiple subjects over a 1-y period to provide a practical approach to assess and perform biologic control testing.MethodsWe performed a single-center, prospective study with 4 healthy subjects, 2 men and 2 women. Subjects performed constant work rate exercise tests for 6 min each at 25-100 W intervals on a computer-controlled cycle ergometer. Data were averaged over the last 120 s at each work rate to reflect stepwise steady-state conditions. Descriptive statistics, including the mean, median, range, SD, and coefficient of variation (CoV) are reported for each individual across the 4 work rates and all repetitions. As these data were normative, z-scores were utilized, and a value greater than ± 1.96 z-scores was used to define significant test variability.ResultsSubjects performed 16-39 biocontrol studies over 1-y. The mean CoV for all subjects in V̇O2 was 6.59%, V̇CO2 was 6.41%, and V̇E was 6.32%. The ± 1.96 z-scores corresponded to a 9.4-18.1% change in V̇O2 , a 9.6-18.1% change in V̇CO2 , and a 9-21.5% change in V̇E across the 4 workloads.ConclusionsWe report long-term variability for steady-state measurement of V̇O2 , V̇CO2 , and V̇E obtained during biocontrol testing. Utilizing ± 1.96 z-scores allows one to determine if a result exceeds expected variability, which may warrant investigation of the instrument.
| S-EPMC9993516 | biostudies-literature
Project description:PurposeInter-individual variability in clinical endpoints and occurrence of potentially severe adverse effects represent an enormous challenge in drug development at all phases of (pre-)clinical research. To ensure patient safety it is important to identify adverse events or critical subgroups within the population as early as possible. Hence, a comprehensive understanding of the processes governing pharmacokinetics and pharmacodynamics is of utmost importance. In this paper we combine Bayesian statistics with detailed mechanistic physiologically-based pharmacokinetic (PBPK) models. On the example of pravastatin we demonstrate that this combination provides a powerful tool to investigate inter-individual variability in groups of patients and to identify clinically relevant homogenous subgroups in an unsupervised approach. Since PBPK models allow the identification of physiological, drug-specific and genotype-specific knowledge separately, our approach supports knowledge-based extrapolation to other drugs or populations.MethodsPBPK models are based on generic distribution models and extensive collections of physiological parameters and allow a mechanistic investigation of drug distribution and drug action. To systematically account for parameter variability within patient populations, a Bayesian-PBPK approach is developed rigorously quantifying the probability of a parameter given the amount of information contained in the measured data. Since these parameter distributions are high-dimensional, a Markov chain Monte Carlo algorithm is used, where the physiological and drug-specific parameters are considered in separate blocks.ResultsConsidering pravastatin pharmacokinetics as an application example, Bayesian-PBPK is used to investigate inter-individual variability in a cohort of 10 patients. Correlation analyses infer structural information about the PBPK model. Moreover, homogeneous subpopulations are identified a posteriori by examining the parameter distributions, which can even be assigned to a polymorphism in the hepatic organ anion transporter OATP1B1.ConclusionsThe presented Bayesian-PBPK approach systematically characterizes inter-individual variability within a population by updating prior knowledge about physiological parameters with new experimental data. Moreover, clinically relevant homogeneous subpopulations can be mechanistically identified. The large scale PBPK model separates physiological and drug-specific knowledge which allows, in combination with Bayesian approaches, the iterative assessment of specific populations by integrating information from several drugs.
| S-EPMC4230716 | biostudies-literature
Project description:Watermelon mosaic virus (WMV) is one of the important Potyviruses that infect cucurbits worldwide. To better understand the population structure of WMV in the United States (U.S.), 57 isolates were collected from cucurbit fields located in nine southern states. The complete coat protein gene of all WMV isolates was cloned, sequenced and compared with 89 reported WMV isolates. The nucleotide and amino acid sequence identities among the U.S. WMV isolates ranged from 88.9 to 99.7% and from 91.5 to 100%, respectively. Phylogenetic analysis revealed that all the U.S. WMV isolates irrespective of their geographic origin or hosts belonged to Group 3. However, the fifty-seven isolates made three clusters in G3, where two clusters were similar to previously reported subgroups EM1 and EM2, and the third cluster, containing nine WMV isolates, formed a distinct subgroup named EM5 in this study. The ratio of non-synonymous to synonymous nucleotide substitution was low indicating the occurrence of negative purifying selection in the CP gene of WMV. Phylogenetic analysis of selected 37 complete genome sequences of WMV isolates also supported the above major grouping. Recombination analysis in the CP genes confirmed various recombinant events, indicating that purifying selection and recombination are the two dominant forces for the evolution of WMV isolates in the U.S.
| S-EPMC8538135 | biostudies-literature
Project description:A survey of doctors working in two large NHS hospitals identified over 120 laboratory tests, imaging investigations and investigational procedures that they considered not to be overused. A common suggestion in this survey was that more training was required. And, this prompted the development of a list of core principles for high-quality, high-value testing. The list can be used as a framework for training and as a reference source. The core principles are: (1) Base testing practices on the best available evidence. (2) Apply the evidence on test performance with careful judgement. (3) Test efficiently. (4) Consider the value (and affordability) of a test before requesting it. (5) Be aware of the downsides and drivers of overdiagnosis. (6) Confront uncertainties. (7) Be patient-centred in your approach. (8) Consider ethical issues. (9) Be aware of normal cognitive limitations and biases when testing. (10) Follow the 'knowledge journey' when teaching and learning these core principles.
| S-EPMC3585491 | biostudies-literature
Project description:IntroductionIn the Investigating the Impact of Alzheimer's Disease Diagnostics in British Columbia (IMPACT-AD BC) study, we aimed to understand how Alzheimer's disease (AD) cerebrospinal fluid (CSF) biomarker testing-used in medical care-impacted medical decision-making (medical utility), personal decision-making (personal utility), and health system economics.MethodsThe study was designed as an observational, longitudinal cohort study. A total of 149 patients were enrolled between February 2019 and July 2021. Patients referred to memory clinics were approached to participate if their dementia specialist ordered AD CSF biomarker testing as part of their routine medical care, and the clinical scenario met the appropriate use criteria for lumbar puncture and AD CSF biomarker testing. For the medical utility pillar, detailed clinical management plans were collected via physician questionnaires pre- and post-biomarker disclosure.ResultsPatients with completed management questionnaires (n = 142) had a median age of 64 (interquartile range: 59-69) years, 48% were female, and 60% had CSF biomarker profiles on the AD continuum. Clinical management changed in 89.4% of cases. AD biomarker testing was associated with decreased need for other diagnostic procedures, including brain imaging (-52.0%) and detailed neuropsychological assessments (-63.2%), increased referrals and counseling (57.0%), and guided AD-related drug prescriptions (+88.4% and -50.0% in biomarker-positive and -negative cases, respectively).DiscussionAD biomarker testing was associated with significant and positive changes in clinical management, including decreased health care resource use, therapy optimization, and increased patient and family member counseling. While certain changes in management were linked to the AD biomarker profile (e.g., referral to clinical trials), the majority of changes were independent of baseline clinical presentation and level of cognitive impairment, demonstrating a broad value for AD biomarker testing in individuals meeting the appropriate use criteria for testing.
| S-EPMC10999950 | biostudies-literature
Project description:The effect of treatment on binary disease outcome can differ across subgroups characterized by other covariates. Testing for the existence of subgroups that are associated with heterogeneous treatment effects can provide valuable insight regarding the optimal treatment recommendation in practice. Our research in this paper is motivated by the question of whether host genetics could modify a vaccine's effect on HIV acquisition risk. To answer this question, we used data from an HIV vaccine trial with a two-phase sampling design and developed a general threshold-based model framework to test for the existence of subgroups associated with the heterogeneity in disease risks, allowing for subgroups based on multivariate covariates. We developed a testing procedure based on maximum of likelihood-ratio statistics over change planes and demonstrated its advantage over alternative methods. We further developed the testing procedure to account for bias sampling of expensive (i.e. resource-intensive to measure) covariates through the incorporation of inverse probability weighting techniques. We used the proposed method to analyze the motivating HIV vaccine trial data. Our proposed testing procedure also has broad applications in epidemiological studies for assessing heterogeneity in disease risk with respect to univariate or multivariate predictors.
| S-EPMC8032557 | biostudies-literature