Dataset Information

What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science.

ABSTRACT: A recent study of the replicability of key psychological findings is a major contribution toward understanding the human side of the scientific process. Despite the careful and nuanced analysis reported, the simple narrative disseminated by the mass, social, and scientific media was that in only 36% of the studies were the original results replicated. In the current study, however, we showed that 77% of the replication effect sizes reported were within a 95% prediction interval calculated using the original effect size. Our analysis suggests two critical issues in understanding replication of psychological studies. First, researchers' intuitive expectations for what a replication should show do not always match with statistical estimates of replication. Second, when the results of original studies are very imprecise, they create wide prediction intervals-and a broad range of replication effects that are consistent with the original estimates. This may lead to effects that replicate successfully, in that replication results are consistent with statistical expectations, but do not provide much information about the size (or existence) of the true effect. In this light, the results of the Reproducibility Project: Psychology can be viewed as statistically consistent with what one might expect when performing a large-scale replication experiment.

SUBMITTER: Patil P

PROVIDER: S-EPMC4968573 | biostudies-literature | 2016 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science.

Patil Prasad P Peng Roger D RD Leek Jeffrey T JT

Perspectives on psychological science : a journal of the Association for Psychological Science 20160701 4

A recent study of the replicability of key psychological findings is a major contribution toward understanding the human side of the scientific process. Despite the careful and nuanced analysis reported, the simple narrative disseminated by the mass, social, and scientific media was that in only 36% of the studies were the original results replicated. In the current study, however, we showed that 77% of the replication effect sizes reported were within a 95% prediction interval calculated using ...[more]

PMID: 27474140

Similar Datasets

Project description:Funding for research is under pressure to be accountable in terms of benefits and translation of research findings into practice and policy. Primary health care research has considerable potential to improve health care in a wide range of settings, but little is known about the extent to which these impacts actually occur. This study examines the impact of individual primary health care research projects on policy and practice from the perspective of Chief Investigators (CIs).The project used an online survey adapted from the Buxton and Hanney Payback Framework to collect information about the impacts that CIs expected and achieved from primary health care research projects funded by Australian national competitive grants.Chief Investigators (CIs) provided information about seventeen completed projects. While no CI expected their project to have an impact in every domain of the framework used in the survey, 76% achieved at least half the impacts they expected. Sixteen projects had published and/or presented their work, 10 projects included 11 doctorate awards in their research capacity domain. All CIs expected their research to lead to further research opportunities with 11 achieving this. Ten CIs achieved their expectation of providing information for policy making but only four reported their research had influenced policy making. However 11 CIs achieved their expectation of providing information for organizational decision making and eight reported their research had influenced organizational decision making.CIs reported that nationally funded primary health care research projects made an impact on knowledge production, staff development and further research, areas within the realm of influence of the research team and within the scope of awareness of the CIs. Some also made an impact on policy and organizational decision-making, and on localized clinical practice and service delivery. CIs reported few broader economic benefits from their research. Routine use of an instrument of this type would facilitate primary health care research funders' determination of the payback for funding of research in this sector.

Dataset Information

What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science.

Publications

What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets