Dataset Information

Using Bounds to Compare the Strength of Exchangeability Assumptions for Internal and External Validity.

ABSTRACT: In the absence of strong assumptions (e.g., exchangeability), only bounds for causal effects can be identified. Here we describe bounds for the risk difference for an effect of a binary exposure on a binary outcome in 4 common study settings: observational studies and randomized studies, each with and without simple random selection from the target population. Through these scenarios, we introduce randomizations for selection and treatment, and the widths of the bounds are narrowed from 2 (the width of the range of the risk difference) to 0 (point identification). We then assess the strength of the assumptions of exchangeability for internal and external validity by comparing their contributions to the widths of the bounds in the setting of an observational study without random selection from the target population. We find that when less than two-thirds of the target population is selected into the study, the assumption of exchangeability for external validity of the risk difference is stronger than that for internal validity. The relative strength of these assumptions should be considered when designing, analyzing, and interpreting observational studies and will aid in determining the best methods for estimating the causal effects of interest.

SUBMITTER: Breskin A

PROVIDER: S-EPMC6601532 | biostudies-literature | 2019 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using Bounds to Compare the Strength of Exchangeability Assumptions for Internal and External Validity.

Breskin Alexander A Westreich Daniel D Cole Stephen R SR Edwards Jessie K JK

American journal of epidemiology 20190701 7

In the absence of strong assumptions (e.g., exchangeability), only bounds for causal effects can be identified. Here we describe bounds for the risk difference for an effect of a binary exposure on a binary outcome in 4 common study settings: observational studies and randomized studies, each with and without simple random selection from the target population. Through these scenarios, we introduce randomizations for selection and treatment, and the widths of the bounds are narrowed from 2 (the w ...[more]

PMID: 30834430

Similar Datasets

Project description:An accurate simulation of the absorption properties is key for assessing the radiative effects of aerosol on meteorology and climate. The representation of how chemical species are mixed inside the particles (the mixing state) is one of the major uncertainty factors in the assessment of these effects. Here we compare aerosol optical properties simulations over Europe and North America, coordinated in the framework of the third phase of the Air Quality Model Evaluation International Initiative (AQMEII), to 1 year of AERONET sunphotometer retrievals, in an attempt to identify a mixing state representation that better reproduces the observed single scattering albedo and its spectral variation. We use a single post-processing tool (FlexAOD) to derive aerosol optical properties from simulated aerosol speciation profiles, and focus on the absorption enhancement of black carbon when it is internally mixed with more scattering material, discarding from the analysis scenes dominated by dust. We found that the single scattering albedo at 440 nm (? 0,440) is on average overestimated (underestimated) by 3-5 % when external (core-shell internal) mixing of particles is assumed, a bias comparable in magnitude with the typical variability of the quantity. The (unphysical) homogeneous internal mixing assumption underestimates ? 0,440 by ~ 14 %. The combination of external and core-shell configurations (partial internal mixing), parameterized using a simplified function of air mass aging, reduces the ? 0,440 bias to -1/-3 %. The black carbon absorption enhancement (E abs) in core-shell with respect to the externally mixed state is in the range 1.8-2.5, which is above the currently most accepted upper limit of ~ 1.5. The partial internal mixing reduces E abs to values more consistent with this limit. However, the spectral dependence of the absorption is not well reproduced, and the absorption Ångström exponent AAE675440 is overestimated by 70-120 %. Further testing against more comprehensive campaign data, including a full characterization of the aerosol profile in terms of chemical speciation, mixing state, and related optical properties, would help in putting a better constraint on these calculations.

Project description:ObjectivesTo assess aspects of the internal validity of recently published cluster randomised trials and explore the reporting of information useful in assessing the external validity of these trials.DesignReview of 34 cluster randomised trials in primary care published in 2004 and 2005 in seven journals (British Medical Journal, British Journal of General Practice, Family Practice, Preventive Medicine, Annals of Internal Medicine, Journal of General Internal Medicine, Pediatrics).Data sourcesNational Library of Medicine (Medline) via PubMed.Data extractionTo assess aspects of internal validity we extracted data on appropriateness of sample size calculations and analyses, methods of identifying and recruiting individual participants, and blinding. To explore reporting of information useful in assessing external validity we extracted data on cluster eligibility, cluster inclusion and retention, cluster generalisability, and the feasibility and acceptability of the intervention to health providers in clusters.Results21 (62%) trials accounted for clustering in sample size calculations and 30 (88%) in the analysis; about a quarter were potentially biased because of procedures surrounding recruitment and identification of patients; individual participants were blind to allocation status in 19 (56%) and outcome assessors were blind in 15 (44%). In almost half the reports, information relating to generalisability of clusters was poorly reported, and in two fifths there was no information about the feasibility and acceptability of the intervention.ConclusionsCluster randomised trials are essential for evaluating certain types of interventions. Issues affecting their internal validity, such as appropriate sample size calculations and analysis, have been widely disseminated and are now better addressed by researchers. Blinding of those identifying and recruiting patients to allocation status is recommended but is not always carried out. There may be fewer barriers to internal validity in trials in which individual participants are not recruited. External validity seems poorly addressed in many trials, yet is arguably as important as internal validity in judging quality as a basis for healthcare intervention.

Project description:BACKGROUND: Controlled clinical trials of health care interventions are either explanatory or pragmatic. Explanatory trials test whether an intervention is efficacious; that is, whether it can have a beneficial effect in an ideal situation. Pragmatic trials measure effectiveness; they measure the degree of beneficial effect in real clinical practice. In pragmatic trials, a balance between external validity (generalizability of the results) and internal validity (reliability or accuracy of the results) needs to be achieved. The explanatory trial seeks to maximize the internal validity by assuring rigorous control of all variables other than the intervention. The pragmatic trial seeks to maximize external validity to ensure that the results can be generalized. However the danger of pragmatic trials is that internal validity may be overly compromised in the effort to ensure generalizability. We are conducting two pragmatic randomized controlled trials on interventions in the management of hypertension in primary care. We describe the design of the trials and the steps taken to deal with the competing demands of external and internal validity. DISCUSSION: External validity is maximized by having few exclusion criteria and by allowing flexibility in the interpretation of the intervention and in management decisions. Internal validity is maximized by decreasing contamination bias through cluster randomization, and decreasing observer and assessment bias, in these non-blinded trials, through baseline data collection prior to randomization, automating the outcomes assessment with 24 hour ambulatory blood pressure monitors, and blinding the data analysis. SUMMARY: Clinical trials conducted in community practices present investigators with difficult methodological choices related to maintaining a balance between internal validity (reliability of the results) and external validity (generalizability). The attempt to achieve methodological purity can result in clinically meaningless results, while attempting to achieve full generalizability can result in invalid and unreliable results. Achieving a creative tension between the two is crucial.

Project description:OBJECTIVE:To conduct the first meta-analysis evaluating the internal and external validity of the sluggish cognitive tempo (SCT) construct as related to or distinct from attention-deficit/hyperactivity disorder (ADHD) and as associated with functional impairment and neuropsychological functioning. METHOD:Electronic databases were searched through September 2015 for studies examining the factor structure and/or correlates of SCT in children or adults. The search procedures identified 73 papers. The core SCT behaviors included across studies, as well as factor loadings and reliability estimates, were reviewed to evaluate internal validity. Pooled correlation effect sizes using random effects models were used to evaluate SCT in relation to external validity domains (i.e., demographics, other psychopathologies, functional impairment, and neuropsychological functioning). RESULTS:Strong support was found for the internal validity of the SCT construct. Specifically, across factor analytic studies including more than 19,000 individuals, 13 SCT items loaded consistently on an SCT factor as opposed to an ADHD factor. Findings also support the reliability (i.e., internal consistency, test-retest reliability, interrater reliability) of SCT. In terms of external validity, there is some indication that SCT may increase with age (r = 0.11) and be associated with lower socioeconomic status (r = 0.10). Modest (potentially negligible) support was found for SCT symptoms being higher in males than females in children (r = 0.05) but not in adults. SCT is more strongly associated with ADHD inattention (r = 0.63 in children, r = 0.72 in adults) than with ADHD hyperactivity-impulsivity (r = 0.32 in children, r = 0.46 in adults), and it likewise appears that SCT is more strongly associated with internalizing symptoms than with externalizing symptoms. SCT is associated with significant global, social, and academic impairment (r = 0.38-0.44). Effects for neuropsychological functioning are mixed, although there is initial support for SCT being associated with processing speed, sustained attention, and metacognitive deficits. CONCLUSION:This meta-analytic review provides strong support for the internal validity of SCT and preliminary support for the external validity of SCT. In terms of diagnostic validity, there is currently not enough evidence to describe SCT in diagnostic terms. Key directions for future research are discussed, including evaluating the conceptualization of SCT as a transdiagnostic construct and the need for longitudinal research.

Dataset Information

Using Bounds to Compare the Strength of Exchangeability Assumptions for Internal and External Validity.

Publications

Using Bounds to Compare the Strength of Exchangeability Assumptions for Internal and External Validity.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets