Controlling false positive rates in mass-multivariate tests for electromagnetic responses.
Ontology highlight
ABSTRACT:
SUBMITTER: Barnes GR
PROVIDER: S-EPMC3092987 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundThe False Discovery Rate (FDR) controls the expected number of false positives among the positive test results. It is not straightforward how to conduct a FDR controlling procedure in experiments with a factorial structure, while at the same time there are between-subjects and within-subjects factors. This is because there are P-values for different tests in one and the same response along with P-values for the same test and different responses.FindingsWe propose a procedure resulting in a single P-value per response, calculated over the tests of all the factorial effects. FDR control can then be based on the set of single P-values.ConclusionsThe proposed procedure is very easy to apply and is recommended for all designs with factors applied at different levels of the randomization, such as cross-over designs with added between-subjects factors.Trial registrationNCT00959790.
| S-EPMC3682898 | biostudies-literature
Project description:The false positive rates (FPR) for surface-based group analysis of cortical thickness, surface area, and volume were evaluated for parametric and non-parametric clusterwise correction for multiple comparisons for a range of smoothing levels and cluster-forming thresholds (CFT) using real data under group assignments that should not yield significant results. For whole cortical surface analysis, thickness showed modest inflation in parametric FPRs above the nominal level (10% versus 5%). Surface area and volume FPRs were much higher (20-30%). In the analysis of interhemispheric thickness asymmetries, FPRs were well controlled by parametric correction, but FPRs for surface area and volume asymmetries were still inflated. In all cases, non-parametric permutation adequately controlled the FPRs. It was found that inflated parametric FPRs were caused by violations in the parametric assumptions, namely a heavier-than-Gaussian spatial correlation. The non-Gaussian spatial correlation originates from anatomical features unique to individuals (e.g., a patch of cortex slightly thicker or thinner than average) and is not a by-product of scanning or processing. Thickness performed better than surface area and volume because thickness does not require a Jacobian correction.
| S-EPMC5857431 | biostudies-literature
Project description:Recent reports of inflated false-positive rates (FPRs) in FMRI group analysis tools by Eklund and associates in 2016 have become a large topic within (and outside) neuroimaging. They concluded that existing parametric methods for determining statistically significant clusters had greatly inflated FPRs ("up to 70%," mainly due to the faulty assumption that the noise spatial autocorrelation function is Gaussian shaped and stationary), calling into question potentially "countless" previous results; in contrast, nonparametric methods, such as their approach, accurately reflected nominal 5% FPRs. They also stated that AFNI showed "particularly high" FPRs compared to other software, largely due to a bug in 3dClustSim. We comment on these points using their own results and figures and by repeating some of their simulations. Briefly, while parametric methods show some FPR inflation in those tests (and assumptions of Gaussian-shaped spatial smoothness also appear to be generally incorrect), their emphasis on reporting the single worst result from thousands of simulation cases greatly exaggerated the scale of the problem. Importantly, FPR statistics depends on "task" paradigm and voxelwise p value threshold; as such, we show how results of their study provide useful suggestions for FMRI study design and analysis, rather than simply a catastrophic downgrading of the field's earlier results. Regarding AFNI (which we maintain), 3dClustSim's bug effect was greatly overstated-their own results show that AFNI results were not "particularly" worse than others. We describe further updates in AFNI for characterizing spatial smoothness more appropriately (greatly reducing FPRs, although some remain >5%); in addition, we outline two newly implemented permutation/randomization-based approaches producing FPRs clustered much more tightly about 5% for voxelwise p ≤ 0.01.
| S-EPMC5399747 | biostudies-literature
Project description:BACKGROUND:Malaria rapid diagnostic tests (RDTs) can produce false positive (FP) results in patients with human African trypanosomiasis and rheumatoid factor (RF), but specificity against other infectious agents and immunological factors is largely unknown. Low diagnostic specificity caused by cross-reactivity may lead to over-estimates of the number of malaria cases and over-use of antimalarial drugs, at the cost of not diagnosing and treating the true underlying condition. METHODS:Data from the WHO Malaria RDT Product Testing Programme was analysed to assess FP rates of 221 RDTs against four infectious agents (Chagas, dengue, Leishmaniasis and Schistosomiasis) and four immunological factors (anti-nuclear antibody, human anti-mouse antibody (HAMA), RF and rapid plasma regain). Only RDTs with a FP rate against clean negative samples less than 10% were included. Paired t-tests were used to compare product-specific FP rates on clean negative samples and samples containing non-Plasmodium infectious agents and immunological factors. RESULTS:Forty (18%) RDTs showed no FP results against any tested infectious agent or immunological factor. In the remaining RDTs significant and clinically relevant increases in FP rates were observed for samples containing HAMA and RF (P<0.001). There were significant correlations between product-matched FP rates for RF and HAMA on all RDT test bands (P<0.001), and FP rates for each infectious agent and immunological factor were also correlated between test bands of combination RDTs (P?0.002). CONCLUSIONS:False positive results against non-Plasmodium infectious agents and immunological factors does not appear to be a universal property of malaria RDTs. However, since many malaria RDTs have elevated FP rates against HAMA and RF positive samples practitioners may need to consider the possibility of false positive results for malaria in patients with conditions that stimulate HAMA or RF.
| S-EPMC5951549 | biostudies-literature
Project description:Two-point linkage analyses of whole genome sequence data are a promising approach to identify rare variants that segregate with complex diseases in large pedigrees because, in theory, the causal variants have been genotyped. We used whole genome sequence data and simulated traits provided by Genetic Analysis Workshop 18 to evaluate the proportion of false-positive findings in a binary trait using classic two-point parametric linkage analysis. False-positive genome-wide significant log of odds (LOD) scores were identified in more than 80% of 50 replicates for a binary phenotype generated by dichotomizing a quantitative trait that was simulated with a polygenic component (that was not based on any of the provided whole genome sequence genotypes). In contrast, when the trait was truly nongenetic (created by randomly assigning affected-unaffected status), the number of false-positive results was well controlled. These results suggest that when using two-point linkage analyses on whole genome sequence data, one should carefully examine regions yielding significant two-point LOD scores with multipoint analysis and that a more stringent significance threshold may be needed.
| S-EPMC4143621 | biostudies-literature
Project description:Issues with inflated false positive rates (FPRs) in brain imaging have recently received significant attention. However, to what extent FPRs present a problem for voxelwise analyses of Positron Emission Tomography (PET) data remains unknown. In this work, we evaluate the FPR using real PET data under group assignments that should yield no significant results after correcting for multiple comparisons. We used data from 159 healthy participants, imaged with the serotonin transporter ([11C]DASB; N = 100) or the 5-HT4 receptor ([11C]SB207145; N = 59). Using this null data, we estimated the FPR by performing 1,000 group analyses with randomly assigned groups of either 10 or 20, for each tracer, and corrected for multiple comparisons using parametric Monte Carlo simulations (MCZ) or non-parametric permutation testing. Our analyses show that for group sizes of 10 or 20, the FPR for both tracers was 5-99% using MCZ, much higher than the expected 5%. This was caused by a heavier-than-Gaussian spatial autocorrelation, violating the parametric assumptions. Permutation correctly controlled the FPR in all cases. In conclusion, either a conservative cluster forming threshold and high smoothing levels, or a non-parametric correction for multiple comparisons should be performed in voxelwise analyses of brain PET data.
| S-EPMC8221774 | biostudies-literature
Project description:Improved procedures, in terms of smaller missed discovery rates (MDR), for performing multiple hypotheses testing with weak and strong control of the family-wise error rate (FWER) or the false discovery rate (FDR) are developed and studied. The improvement over existing procedures such as the Šidák procedure for FWER control and the Benjamini-Hochberg (BH) procedure for FDR control is achieved by exploiting possible differences in the powers of the individual tests. Results signal the need to take into account the powers of the individual tests and to have multiple hypotheses decision functions which are not limited to simply using the individual p-values, as is the case, for example, with the Šidák, Bonferroni, or BH procedures. They also enhance understanding of the role of the powers of individual tests, or more precisely the receiver operating characteristic (ROC) functions of decision processes, in the search for better multiple hypotheses testing procedures. A decision-theoretic framework is utilized, and through auxiliary randomizers the procedures could be used with discrete or mixed-type data or with rank-based nonparametric tests. This is in contrast to existing p-value based procedures whose theoretical validity is contingent on each of these p-value statistics being stochastically equal to or greater than a standard uniform variable under the null hypothesis. Proposed procedures are relevant in the analysis of high-dimensional "large M, small n" data sets arising in the natural, physical, medical, economic and social sciences, whose generation and creation is accelerated by advances in high-throughput technology, notably, but not limited to, microarray technology.
| S-EPMC4091923 | biostudies-literature
Project description:Retention is the most common complication of capsule endoscopy (CE), and is reported to occur in 0-13% of cases. To avoid retention, a PillCam patency capsule (PC) is used in patients with suspected intestinal stenosis. However, a relatively low positive predictive value of the PC examination has been reported previously. The aims of this study were to clarify the accuracy of PC examination and to evaluate clinical factors related to cases of false-positive detection.We performed a retrospective single-center study of 282 consecutive patients referred for PC examination. Patients in which the PC could not pass through the small bowel within 33 h were classified into the 'no patency' group. The 'no patency' group was investigated for evidence of significant stenosis upon further examinations, including CE, double-balloon endoscopy, and small bowel follow-through after PC examination. Clinical factors related to small bowel patency and false-positive cases were evaluated.We included 161 male (57.1%) and 121 female (42.9%) patients with a mean age of 47.5 ± 17.7 years. Of the 282 patients enrolled, 27 patients exhibited 'no patency' upon PC examination. Multivariate analysis showed that clinical factors related to 'no patency' included Crohn's disease, abdominal symptoms, stenosis upon imaging, and previous abdominal surgery. Upon further examination, nine cases in the 'no patency' group had significant stenosis. Sensitivity, specificity, and negative and positive predictive values of PC examination for detecting small bowel stenosis were 93.8%, 96.6%, 99.6%, and 62.5%, respectively, and the only clinical factor related to false-positive cases was constipation (p < 0.05).We found a relatively low positive predictive value of PC examination and that constipation was related to false-positive results. To extend the implications of CE indications, clinical study focusing on these results is expected.
| S-EPMC5557191 | biostudies-other
Project description:BackgroundExercise electrocardiographic stress testing (EST) has historically been validated against the demonstration of obstructive coronary artery disease. However, myocardial ischemia can occur because of coronary microvascular dysfunction (CMD) in the absence of obstructive coronary artery disease.ObjectivesThe aim of this study was to assess the specificity of EST to detect an ischemic substrate against the reference standard of coronary endothelium-independent and endothelium-dependent microvascular function in patients with angina with nonobstructive coronary arteries (ANOCA).MethodsPatients with ANOCA underwent invasive coronary physiological assessment using adenosine and acetylcholine. CMD was defined as impaired endothelium-independent and/or endothelium-dependent function. EST was performed using a standard Bruce treadmill protocol, with ischemia defined as the appearance of ≥0.1-mV ST-segment depression 80 ms from the J-point on electrocardiography. The study was powered to detect specificity of ≥91%.ResultsA total of 102 patients were enrolled (65% women, mean age 60 ± 8 years). Thirty-two patients developed ischemia (ischemic group) during EST, whereas 70 patients did not (nonischemic group); both groups were phenotypically similar. Ischemia during EST was 100% specific for CMD. Acetylcholine flow reserve was the strongest predictor of ischemia during exercise. Using endothelium-independent and endothelium-dependent microvascular dysfunction as the reference standard, the false positive rate of EST dropped to 0%.ConclusionsIn patients with ANOCA, ischemia on EST was highly specific of an underlying ischemic substrate. These findings challenge the traditional belief that EST has a high false positive rate.
| S-EPMC10790243 | biostudies-literature
Project description:Characterization of endogenous neuropeptides produced from post-translational proteolytic processing of precursor proteins is a demanding task. A variety of complex prohormone processing steps generate molecular diversity from neuropeptide prohormones, making in silico neuropeptide discovery difficult. In addition, the wide range of endogenous peptide concentrations as well as significant peptide complexity further challenge the structural characterization of neuropeptides. Liquid chromatography-mass spectrometry (MS), performed in conjunction with bioinformatics, allows for high-throughput characterization of peptides. Mass analyzers and molecular dissociation techniques render specific characteristics to the acquired data and thus, influence the analysis of the MS data using bioinformatic algorithms for follow-up peptide identification. Here we evaluated the efficacy of several distinct peptidomic workflows using two mass spectrometers, the Thermo Orbitrap Fusion Tribrid and Bruker Impact HD UHR-QqTOF, for confident peptide discovery and characterization. We compared the results in several categories, including the numbers of identified peptides, full-length mature neuropeptides among all identifications, and precursor proteins mapped by the identified peptides. We also characterized the peptide false discovery rate (FDR) based on the occurrence of amidation, a known post-translational modification (PTM) that has been shown to require the presence of a C-terminal glycine. Thus, amidation events without a preceding glycine were considered false-positive amidation assignments. We compared the FDR calculated by the search engine used here to the minimum FDR estimated via false amidation assignments. The search engine severely underestimated the rate of false PTM assignments among the identified peptides, regardless of the specific MS platform used.
| S-EPMC5839655 | biostudies-literature