Dataset Information

Effects of non-differential exposure misclassification on false conclusions in hypothesis-generating studies.

ABSTRACT: Despite the theoretical success of obviating the need for hypothesis-generating studies, they live on in epidemiological practice. Cole asserted that "… there is boundless number of hypotheses that could be generated, nearly all of them wrong" and urged us to focus on evaluating "credibility of hypothesis". Adopting a Bayesian approach, we put this elegant logic into quantitative terms at the study planning stage for studies where the prior belief in the null hypothesis is high (i.e., "hypothesis-generating" studies). We consider not only type I and II errors (as is customary) but also the probabilities of false positive and negative results, taking into account typical imperfections in the data. We concentrate on a common source of imperfection in the data: non-differential misclassification of binary exposure classifier. In context of an unmatched case-control study, we demonstrate-both theoretically and via simulations-that although non-differential exposure misclassification is expected to attenuate real effect estimates, leading to the loss of ability to detect true effects, there is also a concurrent increase in false positives. Unfortunately, most investigators interpret their findings from such work as being biased towards the null rather than considering that they are no less likely to be false signals. The likelihood of false positives dwarfed the false negative rate under a wide range of studied settings. We suggest that instead of investing energy into understanding credibility of dubious hypotheses, applied disciplines such as epidemiology, should instead focus attention on understanding consequences of pursuing specific hypotheses, while accounting for the probability that the observed "statistically significant" association may be qualitatively spurious.

SUBMITTER: Burstyn I

PROVIDER: S-EPMC4211016 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Effects of non-differential exposure misclassification on false conclusions in hypothesis-generating studies.

Burstyn Igor I Yang Yunwen Y Schnatter A Robert AR

International journal of environmental research and public health 20141021 10

Despite the theoretical success of obviating the need for hypothesis-generating studies, they live on in epidemiological practice. Cole asserted that "… there is boundless number of hypotheses that could be generated, nearly all of them wrong" and urged us to focus on evaluating "credibility of hypothesis". Adopting a Bayesian approach, we put this elegant logic into quantitative terms at the study planning stage for studies where the prior belief in the null hypothesis is high (i.e., "hypothesi ...[more]

PMID: 25337942

Similar Datasets

Project description:IntroductionAdjudication of the primary outcome in randomised trials is thought to control misclassification. We investigated the amount of misclassification needed before adjudication changed the primary trial results.Patients (or materials) and methods: We included data from five randomised stroke trials. Differential misclassification was introduced for each primary outcome until the estimated treatment effect was altered. This was simulated 1000 times. We calculated the between-simulation mean proportion of participants that needed to be differentially misclassified to alter the treatment effect. In addition, we simulated hypothetical trials with a binary outcome and varying sample size (1000-10,000), overall event rate (10%-50%) and treatment effect (0.67-0.90). We introduced non-differential misclassification until the treatment effect was non-significant at 5% level.ResultsFor the five trials, the range of unweighted kappa values were reduced from 0.89-0.97 to 0.65-0.85 before the treatment effect was altered. This corresponded to 2.1%-6% of participants misclassified differentially for trials with a binary outcome. For the hypothetical trials, those with a larger sample size, stronger treatment effect and overall event rate closer to 50% needed a higher proportion of events non-differentially misclassified before the treatment effect became non-significant.DiscussionWe found that only a small amount of differential misclassification was required before adjudication altered the primary trial results, whereas a considerable proportion of participants needed to be misclassified non-differentially before adjudication changed trial conclusions. Given that differential misclassification should not occur in trials with sufficient blinding, these results suggest that central adjudication is of most use in studies with unblinded outcome assessment.ConclusionFor trials without adequate blinding, central adjudication is vital to control for differential misclassification. However, for large blinded trials, adjudication is of less importance and may not be necessary.

Project description:BackgroundThe growing interest in research on the health effects of near-highway air pollutants requires an assessment of potential sources of error in exposure assignment techniques that rely on residential proximity to roadways.MethodsWe compared the amount of positional error in the geocoding process for three different data sources (parcels, TIGER and StreetMap USA) to a "gold standard" residential geocoding process that used ortho-photos, large multi-building parcel layouts or large multi-unit building floor plans. The potential effect of positional error for each geocoding method was assessed as part of a proximity to highway epidemiological study in the Boston area, using all participants with complete address information (N = 703). Hourly time-activity data for the most recent workday/weekday and non-workday/weekend were collected to examine time spent in five different micro-environments (inside of home, outside of home, school/work, travel on highway, and other). Analysis included examination of whether time-activity patterns were differentially distributed either by proximity to highway or across demographic groups.ResultsMedian positional error was significantly higher in street network geocoding (StreetMap USA = 23 m; TIGER = 22 m) than parcel geocoding (8 m). When restricted to multi-building parcels and large multi-unit building parcels, all three geocoding methods had substantial positional error (parcels = 24 m; StreetMap USA = 28 m; TIGER = 37 m). Street network geocoding also differentially introduced greater amounts of positional error in the proximity to highway study in the 0-50 m proximity category. Time spent inside home on workdays/weekdays differed significantly by demographic variables (age, employment status, educational attainment, income and race). Time-activity patterns were also significantly different when stratified by proximity to highway, with those participants residing in the 0-50 m proximity category reporting significantly more time in the school/work micro-environment on workdays/weekdays than all other distance groups.ConclusionsThese findings indicate the potential for both differential and non-differential exposure misclassification due to geocoding error and time-activity patterns in studies of highway proximity. We also propose a multi-stage manual correction process to minimize positional error. Additional research is needed in other populations and geographic settings.

Dataset Information

Effects of non-differential exposure misclassification on false conclusions in hypothesis-generating studies.

Publications

Effects of non-differential exposure misclassification on false conclusions in hypothesis-generating studies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets