Dataset Information

Internal conceptual replications do not increase independent replication success.

ABSTRACT: Recently, many psychological effects have been surprisingly difficult to reproduce. This article asks why, and investigates whether conceptually replicating an effect in the original publication is related to the success of independent, direct replications. Two prominent accounts of low reproducibility make different predictions in this respect. One account suggests that psychological phenomena are dependent on unknown contexts that are not reproduced in independent replication attempts. By this account, internal replications indicate that a finding is more robust and, thus, that it is easier to independently replicate it. An alternative account suggests that researchers employ questionable research practices (QRPs), which increase false positive rates. By this account, the success of internal replications may just be the result of QRPs and, thus, internal replications are not predictive of independent replication success. The data of a large reproducibility project support the QRP account: replicating an effect in the original publication is not related to independent replication success. Additional analyses reveal that internally replicated and internally unreplicated effects are not very different in terms of variables associated with replication success. Moreover, social psychological effects in particular appear to lack any benefit from internal replications. Overall, these results indicate that, in this dataset at least, the influence of QRPs is at the heart of failures to replicate psychological findings, especially in social psychology. Variable, unknown contexts appear to play only a relatively minor role. I recommend practical solutions for how QRPs can be avoided.

SUBMITTER: Kunert R

PROVIDER: S-EPMC5050250 | biostudies-literature | 2016 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Internal conceptual replications do not increase independent replication success.

Kunert Richard R

Psychonomic bulletin & review 20161001 5

Recently, many psychological effects have been surprisingly difficult to reproduce. This article asks why, and investigates whether conceptually replicating an effect in the original publication is related to the success of independent, direct replications. Two prominent accounts of low reproducibility make different predictions in this respect. One account suggests that psychological phenomena are dependent on unknown contexts that are not reproduced in independent replication attempts. By this ...[more]

PMID: 27068542

Similar Datasets

Project description:BackgroundScreening for colorectal, breast, and cervical cancer has been shown to reduce mortality; however, not all men and women are screened in the USA. Further, there are disparities in screening uptake by people from racial and ethnic minority groups, people with low income, people who lack health insurance, and those who lack access to care. The Centers for Disease Control and Prevention funds two programs-the Colorectal Cancer Control Program and the National Breast and Cervical Cancer Early Detection Program-to help increase cancer screenings among groups that have been economically and socially marginalized. The goal of this manuscript is to describe how programs and their partners integrate evidence-based interventions (e.g., patient reminders) and supporting activities (e.g., practice facilitation to optimize electronic medical records) across colorectal, breast, and cervical cancer screenings, and we suggest research areas based on implementation science.MethodsWe conducted an exploratory assessment using qualitative and quantitative data to describe implementation of integrated interventions and supporting activities for cancer screening. We conducted 10 site visits and follow-up telephone interviews with health systems and their partners to inform the integration processes. We developed a conceptual model to describe the integration processes and reviewed screening recommendations of the United States Preventive Services Task Force to illustrate challenges in integration. To identify factors important in program implementation, we asked program implementers to rank domains and constructs of the Consolidated Framework for Implementation Research.ResultsHealth systems integrated interventions for all screenings across single and multiple levels. Although potentially efficient, there were challenges due to differing eligibility of screenings by age, gender, frequency, and location of services. Program implementers ranked complexity, cost, implementation climate, and engagement of appropriate staff in implementation among the most important factors to success.ConclusionIntegrating interventions and supporting activities to increase uptake of cancer screenings could be an effective and efficient approach, but we currently do not have the evidence to recommend widescale adoption. Detailed multilevel measures related to process, screening, and implementation outcomes, and cost are required to evaluate integrated programs. Systematic studies can help to ascertain the benefits of integrating interventions and supporting activities for multiple cancer screenings, and we suggest research areas that might address current gaps in the literature.

Project description:Hormonal transitions across the menstrual cycle may modulate human reward processing and reinforcement learning, but previous results were contradictory. Studies assessed relatively small samples (n < 30) and exclusively used within-subject designs to compare women in hormonally distinct menstrual cycle phases. This increased the risk of sporadic findings and results may have been disproportionally affected by expectancy effects. Also, replication studies are widely missing, which currently precludes any reliable inferences. The present study was intended as a conceptual replication of a previous study [(1), Neuropsychologia 84; n = 15]. There, we had observed a reduction in avoidance learning capacity when women were in the high estradiol state of the late follicular phase as compared to the mid luteal phase with enhanced progesterone influence. These results conformed to the idea that estradiol and progesterone may antagonistically modulate dopaminergic transmission as a dopamine agonist and antagonist, respectively. Heightened progesterone in the luteal phase thereby supported the ability to learn from the negative outcomes of one's actions, while the follicular rise in estradiol interfered with this capacity. Here, we re-examined the above described within-subject difference between the follicular and the luteal phase in a between-subjects design. Seventy-five women were tested once with a probabilistic feedback learning task, while being either in the follicular (36 women) or luteal phase (39 women), and were compared for phase-related differences in behavior. Secondly, we combined the new data with data from three previous studies from our laboratory that used the same task and menstrual cycle phases. This meta-analysis included only data from the first test day, free of any biasing expectancy effects. Both analyses demonstrated the consistency of the decline in avoidance learning in the follicular relative to the luteal phase. We also showed that this decline reliably occurred in all of the included samples. Altogether, these results provide evidence for the consistency of a behavioral difference and its apparent association with a transient change in hormonal state that occurs in the natural menstrual cycle. Our findings may also open new avenues for the development of reliable between-subjects test protocols in menstrual cycle research.

Dataset Information

Internal conceptual replications do not increase independent replication success.

Publications

Internal conceptual replications do not increase independent replication success.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets