Dataset Information

A Bayesian Perspective on the Reproducibility Project: Psychology.

ABSTRACT: We revisit the results of the recent Reproducibility Project: Psychology by the Open Science Collaboration. We compute Bayes factors-a quantity that can be used to express comparative evidence for an hypothesis but also for the null hypothesis-for a large subset (N = 72) of the original papers and their corresponding replication attempts. In our computation, we take into account the likely scenario that publication bias had distorted the originally published results. Overall, 75% of studies gave qualitatively similar results in terms of the amount of evidence provided. However, the evidence was often weak (i.e., Bayes factor < 10). The majority of the studies (64%) did not provide strong evidence for either the null or the alternative hypothesis in either the original or the replication, and no replication attempts provided strong evidence in favor of the null. In all cases where the original paper provided strong evidence but the replication did not (15%), the sample size in the replication was smaller than the original. Where the replication provided strong evidence but the original did not (10%), the replication sample size was larger. We conclude that the apparent failure of the Reproducibility Project to replicate many target effects can be adequately explained by overestimation of effect sizes (or overestimation of evidence against the null hypothesis) due to small sample sizes and publication bias in the psychological literature. We further conclude that traditional sample sizes are insufficient and that a more widespread adoption of Bayesian methods is desirable.

SUBMITTER: Etz A

PROVIDER: S-EPMC4769355 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Bayesian Perspective on the Reproducibility Project: Psychology.

Etz Alexander A Vandekerckhove Joachim J

PloS one 20160226 2

We revisit the results of the recent Reproducibility Project: Psychology by the Open Science Collaboration. We compute Bayes factors-a quantity that can be used to express comparative evidence for an hypothesis but also for the null hypothesis-for a large subset (N = 72) of the original papers and their corresponding replication attempts. In our computation, we take into account the likely scenario that publication bias had distorted the originally published results. Overall, 75% of studies gave ...[more]

PMID: 26919473

Similar Datasets

Project description:From an evolutionary perspective, musical behavior such as playing an instrument can be considered as part of an individual's courting behavior. Playing a musical instrument or singing might fulfill a function similar to that of a bird's colored feathers: attracting attention. Therefore, musicians may be rated as more attractive than non-musicians. In an online survey, 137 volunteers (95 female) with ages ranging from 16 to 39 years rated the attractiveness of fictitious persons of the opposite sex described in short verbal profiles. These profiles differed with respect to whether the described person made music or not. Additionally, the musicians' profiles varied with regard to whether the described person played music or sang in public or in private only. Results show that musicians' profiles were not generally rated as more attractive than non-musicians', but attractiveness did vary according to setting: private musicians were rated as most attractive, followed by non-musicians and public musicians. Furthermore, results indicate that participants who played a musical instrument or sang themselves gave higher ratings to profiles of musicians. But for participants who do not make music themselves, higher attractiveness ratings for musicians playing instruments or sing in private settings were found. These results indicate that the impression of sharing a common interest (making music) and furthermore making music in private instrumental settings seems to make people attractive to other people. No additional support for the sexual selection hypotheses for the evolution of music was provided by the current results. The musical status of the rater affected his or her judgements, with musicians rating other people as more attractive if they share the common interest in making music. Not the display of being a musician seems to be critical for attractiveness ratings but the perceived or imagined similarity by the rater created by information on musicality, fostering the theoretical significance of the communication aspect of music.

Dataset Information

A Bayesian Perspective on the Reproducibility Project: Psychology.

Publications

A Bayesian Perspective on the Reproducibility Project: Psychology.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets