Correction: Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature.
Ontology highlight
ABSTRACT:
SUBMITTER: Szucs D
PROVIDER: S-EPMC7935231 | biostudies-literature | 2021 Mar
REPOSITORIES: biostudies-literature
Publications
Correction: Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature.
Szucs Denes D Ioannidis John P A JPA
PLoS biology 20210305 3
[This corrects the article DOI: 10.1371/journal.pbio.2000797.]. ...[more]
Similar Datasets
Project description:We have empirically assessed the distribution of published effect sizes and estimated power by analyzing 26,841 statistical records from 3,801 cognitive neuroscience and psychology papers published recently. The reported median effect size was D = 0.93 (interquartile range: 0.64-1.46) for nominally statistically significant results and D = 0.24 (0.11-0.42) for nonsignificant results. Median power to detect small, medium, and large effects was 0.12, 0.44, and 0.73, reflecting no improvement through the past half-century. This is so because sample sizes have remained small. Assuming similar true effect sizes in both disciplines, power was lower in cognitive neuroscience than in psychology. Journal impact factors negatively correlated with power. Assuming a realistic range of prior probabilities for null hypotheses, false report probability is likely to exceed 50% for the whole literature. In light of our findings, the recently reported low replication success in psychology is realistic, and worse performance may be expected for cognitive neuroscience.
| S-EPMC5333800 | biostudies-literature
Project description:When designing experimental studies with human participants, experimenters must decide how many trials each participant will complete, as well as how many participants to test. Most discussion of statistical power (the ability of a study design to detect an effect) has focused on sample size, and assumed sufficient trials. Here we explore the influence of both factors on statistical power, represented as a 2-dimensional plot on which iso-power contours can be visualized. We demonstrate the conditions under which the number of trials is particularly important, that is, when the within-participant variance is large relative to the between-participants variance. We then derive power contour plots using existing data sets for 8 experimental paradigms and methodologies (including reaction times, sensory thresholds, fMRI, MEG, and EEG), and provide example code to calculate estimates of the within- and between-participants variance for each method. In all cases, the within-participant variance was larger than the between-participants variance, meaning that the number of trials has a meaningful influence on statistical power in commonly used paradigms. An online tool is provided (https://shiny.york.ac.uk/powercontours/) for generating power contours, from which the optimal combination of trials and participants can be calculated when designing future studies. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
| S-EPMC8329985 | biostudies-literature
Project description:We review empirical research on (social) psychology of morality to identify which issues and relations are well documented by existing data and which areas of inquiry are in need of further empirical evidence. An electronic literature search yielded a total of 1,278 relevant research articles published from 1940 through 2017. These were subjected to expert content analysis and standardized bibliometric analysis to classify research questions and relate these to (trends in) empirical approaches that characterize research on morality. We categorize the research questions addressed in this literature into five different themes and consider how empirical approaches within each of these themes have addressed psychological antecedents and implications of moral behavior. We conclude that some key features of theoretical questions relating to human morality are not systematically captured in empirical research and are in need of further investigation.
| S-EPMC6791030 | biostudies-literature
Project description:Attention is the important ability to flexibly control limited computational resources. It has been studied in conjunction with many other topics in neuroscience and psychology including awareness, vigilance, saliency, executive control, and learning. It has also recently been applied in several domains in machine learning. The relationship between the study of biological attention and its use as a tool to enhance artificial neural networks is not always clear. This review starts by providing an overview of how attention is conceptualized in the neuroscience and psychology literature. It then covers several use cases of attention in machine learning, indicating their biological counterparts where they exist. Finally, the ways in which artificial attention can be further inspired by biology for the production of complex and integrative systems is explored.
| S-EPMC7177153 | biostudies-literature
Project description:Prior researchers have shown that the brain has a remarkable ability for adapting to environmental changes. The positive effects of such neural plasticity include enhanced functioning in specific cognitive domains and shifts in cortical representation following naturally occurring cases of sensory deprivation; however, maladaptive changes in brain function and development owing to early developmental adversity and stress have also been well documented. Researchers examining enriched rearing environments in animals have revealed the potential for inducing positive brain plasticity effects and have helped to popularize methods for training the brain to reverse early brain deficits or to boost normal cognitive functioning. In this article, two classes of empirically based methods of brain training in children are reviewed and critiqued: laboratory-based, mental process training paradigms and ecological interventions based upon neurocognitive conceptual models. Given the susceptibility of executive function disruption, special attention is paid to training programs that emphasize executive function enhancement. In addition, a third approach to brain training, aimed at tapping into compensatory processes, is postulated. Study results showing the effectiveness of this strategy in the field of neurorehabilitation and in terms of naturally occurring compensatory processing in human aging lend credence to the potential of this approach. (PsycINFO Database Record (c) 2012 APA, all rights reserved).
| S-EPMC3335430 | biostudies-literature
Project description:To determine the reproducibility of psychological meta-analyses, we investigated whether we could reproduce 500 primary study effect sizes drawn from 33 published meta-analyses based on the information given in the meta-analyses, and whether recomputations of primary study effect sizes altered the overall results of the meta-analysis. Results showed that almost half (k = 224) of all sampled primary effect sizes could not be reproduced based on the reported information in the meta-analysis, mostly because of incomplete or missing information on how effect sizes from primary studies were selected and computed. Overall, this led to small discrepancies in the computation of mean effect sizes, confidence intervals and heterogeneity estimates in 13 out of 33 meta-analyses. We provide recommendations to improve transparency in the reporting of the entire meta-analytic process, including the use of preregistration, data and workflow sharing, and explicit coding practices.
| S-EPMC7252608 | biostudies-literature
Project description:The validity of scientific findings may be challenged by the replicability crisis (or cases of fraud), which may result not only in a loss of trust within society but may also lead to wrong or even harmful policy or medical decisions. The question is: how reliable are scientific results that are reported as statistically significant, and how does this reliability develop over time? Based on 35,515 papers in psychology published between 1975 and 2017 containing 487,996 test values, this article empirically examines the statistical power, publication bias, and p-hacking, as well as the false discovery rate. Assuming constant true effects, the statistical power was found to be lower than the suggested 80% except for large underlying true effects (d = 0.8) and increased only slightly over time. Also, publication bias and p-hacking were found to be substantial. The share of false discoveries among all significant results was estimated at 17.7%, assuming a proportion θ = 50% of all hypotheses being true and assuming that p-hacking is the only mechanism generating a higher proportion of just significant results compared to just nonsignificant results. As the analyses rely on multiple assumptions that cannot be tested, alternative scenarios were laid out, again resulting in the rather optimistic result that although research results may suffer from low statistical power and publication selection bias, most of the results reported as statistically significant may contain substantial results, rather than statistical artifacts.
| S-EPMC10581498 | biostudies-literature
Project description:Science is self-correcting, or so the adage goes, but to what extent is that indeed the case? Answering this question requires careful consideration of the various approaches to achieve the collective goal of self-correction. One of the most straightforward mechanisms is individual self-correction: researchers rectifying their own mistakes by publishing a correction notice. Although it offers an efficient route to correcting the scientific record, it has received little to no attention from a metascientific point of view. We aim to fill this void by analysing the content of correction notices published from 2010 until 2018 in the three psychology journals featuring the highest number of corrections over that timespan based on the Scopus database (i.e. Psychological Science with N = 58, Frontiers in Psychology with N = 99 and Journal of Affective Disorders with N = 57). More concretely, we examined which aspects of the original papers were affected (e.g. hypotheses, data-analyses, metadata such as author order, affiliations, funding information etc.) as well as the perceived implications for the papers' main findings. Our exploratory analyses showed that many corrections involved inconsequential errors. Furthermore, authors rarely revised their conclusions, even though several corrections concerned changes to the results. We conclude with a discussion of current policies, and suggest ways to improve upon the present situation by (i) preventing mistakes, and (ii) transparently rectifying those mistakes that do find their way into the literature.
| S-EPMC7657932 | biostudies-literature
Project description:Successful accumulation of knowledge is critically dependent on the ability to verify and replicate every part of scientific conduct. However, such principles are difficult to enact when researchers continue to resort on ad-hoc workflows and with poorly maintained code base. In this paper I examine the needs of neuroscience and psychology community, and introduce psychopy_ext, a unifying framework that seamlessly integrates popular experiment building, analysis and manuscript preparation tools by choosing reasonable defaults and implementing relatively rigid patterns of workflow. This structure allows for automation of multiple tasks, such as generated user interfaces, unit testing, control analyses of stimuli, single-command access to descriptive statistics, and publication quality plotting. Taken together, psychopy_ext opens an exciting possibility for a faster, more robust code development and collaboration for researchers.
| S-EPMC3894454 | biostudies-other
Project description:This article examines the relation of contemplative exercises with enhancement of students' experience during neuroscience studies. Short contemplative exercises inspired by the Buddhist tradition of self-inquiry were introduced in an undergraduate neuroscience course for psychology students. At the start of the class, all students were asked to participate in short "personal brain investigations" relevant to the topic presented. These investigations were aimed at bringing stable awareness to a specific perceptual, emotional, attentional, or cognitive process and observing it in a non-judgmental, non-personal way. In addition, students could choose to participate, for bonus credit, in a longer exercise designed to expand upon the weekly class activity. In the exercise, students continued their "personal brain investigations" for 10 min a day, 4 days a week. They wrote "lab reports" on their daily observations, obtained feedback from the teacher, and at the end of the year reviewed their reports and reflected upon their experiences during the semester. Out of 265 students, 102 students completed the bonus track and their final reflections were analyzed using qualitative methodology. In addition, 91 of the students answered a survey at the end of the course, 43 students participated in a quiz 1 year after course graduation, and the final grades of all students were collected and analyzed. Overall, students reported satisfaction from the exercises and felt they contributed to their learning experience. In the 1-year follow-up, the bonus-track students were significantly more likely than their peers to remember class material. The qualitative analysis of bonus-track students' reports revealed that the bonus-track process elicited positive feelings, helped students connect with class material and provided them with personal insights. In addition, students acquired contemplative skills, such as increased awareness and attention, non-judgmental attitudes, and better stress-management abilities. We provide examples of "personal brain investigations" and discuss limitations of introducing a contemplative approach.
| S-EPMC4235268 | biostudies-literature