Researchers' Intuitions About Power in Psychological Research.
Ontology highlight
ABSTRACT:
SUBMITTER: Bakker M
PROVIDER: S-EPMC4976648 | biostudies-literature | 2016 Aug
REPOSITORIES: biostudies-literature
Publications
Researchers' Intuitions About Power in Psychological Research.
Bakker Marjan M Hartgerink Chris H J CH Wicherts Jelte M JM van der Maas Han L J HL
Psychological science 20160628 8
Many psychology studies are statistically underpowered. In part, this may be because many researchers rely on intuition, rules of thumb, and prior practice (along with practical considerations) to determine the number of subjects to test. In Study 1, we surveyed 291 published research psychologists and found large discrepancies between their reports of their preferred amount of power and the actual power of their studies (calculated from their reported typical cell size, typical effect size, and ...[more]
Similar Datasets
Project description:BackgroundReports of questionable or detrimental research practices (QRPs) call into question the reliability of scientific evidence and the trustworthiness of research. A critical component of the research ecosystem is the organization within which research takes place. We conducted a survey to explore the attitudes and beliefs of European and American researchers about the organisations in which they work, their own research practices and their attitudes towards research integrity and research integrity policies.MethodsWe administered an online survey (International Research Integrity Survey (IRIS)) to 2,300 active researchers based in the US and 45,000 in Europe (including UK, Norway, Iceland and Switzerland). We employed a stratified probability sample of the authors of research articles published between 2016 and 2020 included in Clarivate's Web of Science citation database. Coverage includes researchers in the humanities, social sciences, natural sciences and medical sciences, who hold at least a master's level degree.ResultsIn comparison to researchers in the US, European researchers admit to more QRPs and are less confident in maintaining high research integrity (RI) standards. In the US and Europe, many researchers judge their organization to fall short of best RI practice. All researchers recognize the benefits of RI, reliable knowledge and the trust of colleagues and the public, and there is support for RI training particularly among Europeans.ConclusionTo create and maintain a culture of integrity in scientific research, a collective commitment from researchers, their institutions and funders is needed. Researchers rely on many channels of communication about research integrity and thus the involvement of many different participants in the research system is required to make improvements. Policies must be developed to reinforce best practice rather than being seen as an irrelevance to the real business of research.
| S-EPMC10349267 | biostudies-literature
Project description:In this paper, we show how Bayes' theorem can be used to better understand the implications of the 36% reproducibility rate of published psychological findings reported by the Open Science Collaboration. We demonstrate a method to assess publication bias and show that the observed reproducibility rate was not consistent with an unbiased literature. We estimate a plausible range for the prior probability of this body of research, suggesting expected statistical power in the original studies of 48-75%, producing (positive) findings that were expected to be true 41-62% of the time. Publication bias was large, assuming a literature with 90% positive findings, indicating that negative evidence was expected to have been observed 55-98 times before one negative result was published. These findings imply that even when studied associations are truly NULL, we expect the literature to be dominated by statistically significant findings.
| S-EPMC6170554 | biostudies-literature
Project description:Background:There is increasing evidence that research misbehaviour is common, especially the minor forms. Previous studies on research misbehaviour primarily focused on biomedical and social sciences, and evidence from natural sciences and humanities is scarce. We investigated what academic researchers in Amsterdam perceived to be detrimental research misbehaviours in their respective disciplinary fields. Methods:We used an explanatory sequential mixed methods design. First, survey participants from four disciplinary fields rated perceived frequency and impact of research misbehaviours from a list of 60. We then combined these into a top five ranking of most detrimental research misbehaviours at the aggregate level, stratified by disciplinary field. Second, in focus group interviews, participants from each academic rank and disciplinary field were asked to reflect on the most relevant research misbehaviours for their disciplinary field. We used participative ranking methodology inducing participants to obtain consensus on which research misbehaviours are most detrimental. Results:In total, 1080 researchers completed the survey (response rate: 15%) and 61 participated in the focus groups (3 three to 8 eight researchers per group). Insufficient supervision consistently ranked highest in the survey regardless of disciplinary field and the focus groups confirmed this. Important themes in the focus groups were insufficient supervision, sloppy science, and sloppy peer review. Biomedical researchers and social science researchers were primarily concerned with sloppy science and insufficient supervision. Natural sciences and humanities researchers discussed sloppy reviewing and theft of ideas by reviewers, a form of plagiarism. Focus group participants further provided examples of particular research misbehaviours they were confronted with and how these impacted their work as a researcher. Conclusion:We found insufficient supervision and various forms of sloppy science to score highly on aggregate detrimental impact throughout all disciplinary fields. Researchers from the natural sciences and humanities also perceived nepotism to be of major impact on the aggregate level. The natural sciences regarded fabrication of data of major impact as well. The focus group interviews helped to understand how researchers interpreted 'insufficient supervision'. Besides, the focus group participants added insight into sloppy science in practice. Researchers from the natural sciences and humanities added new research misbehaviours concerning their disciplinary fields to the list, such as the stealing of ideas before publication. This improves our understanding of research misbehaviour beyond the social and biomedical fields.
| S-EPMC6886174 | biostudies-literature
Project description:With the increase in patient and consumer activism through the late twentieth century and into this century, patient roles in research evolved into a new model of research engagement, with patients serving as active advisors and co-leading or leading clinical research. By requiring active engagement of patients and other stakeholders, several government research funders have advanced this model, particularly in Canada, the United States (US), United Kingdom (UK), and Australia. A consortium of individuals from these countries formed a Multi-Stakeholder Engagement (MuSE) consortium to examine critical issues in engaged research, establish consensus on definitions, and provide guidance for the field, beginning with an overview of how to involve stakeholders in health research (Concannon et al. J Gen Intern Med. 2019;34(3):458-463) and continuing here with an examination of definitions of research engagement. The political and advocacy roots of engaged research are reflected in definitions. Engagement is conceptualized with reference to research project goals, from informing specific clinical decisions to informing health-system level decisions. Political and cultural differences across countries are evident. Some of these government funders focus on empirical rather than ethical rationales. In countries with centralized health technology assessment, the link between societal values and engaged research is explicit. Ethical rationales for engagement are explicit in most of the published literature on research engagement. Harmonization of definitions is recommended so that research engagement elements, methods, and outcomes and impacts can be clearly examined and understood, and so that the field of research engagement can proceed from a clear conceptual foundation. Specific recommendations for terminology definitions are provided. Placing engaged research on a continuum from specific clinical decisions to more global public and social justice concerns clarifies the type of engaged research, supports appropriate comparisons, and improves the rigor of engaged research methods. The results help identify knowledge gaps in this growing field.
| S-EPMC7048327 | biostudies-literature
Project description:Few questions in science are as controversial as the origins of knowledge. Whether knowledge (e.g., "objects are cohesive") is partly innate has been debated for centuries. Here, we ask whether our difficulties with innate knowledge could be grounded in human cognition itself. In eight experiments, we compared reasoning about the innateness of traits that capture knowledge (cognitive traits) with noncognitive (sensorimotor and emotive) traits. Experiments 1-4 examined adult and infant traits; Experiment 5 presented detailed descriptions of published infant experiments. Results showed that people viewed cognitive traits as less likely to be innate in humans-the stronger the association with "thinking," the lower the rating for "innateness." Experiments 6-8 explored human, bird, and alien traits that were presented as innate. Participants, however, still considered cognitive traits as less likely to emerge spontaneously (i.e., be innate). These results show that people are selectively biased in reasoning about the origins of knowledge.
| S-EPMC8412331 | biostudies-literature
Project description:Authorship is commonly used as the basis for the measurement of research productivity. It influences career progression and rewards, making it a valued commodity in a competitive scientific environment. To better understand authorship practices amongst collaborative teams, this study surveyed authors on collaborative journal articles published between 2011 and 2015. Of the 8364 respondents, 1408 responded to the final open-ended question, which solicited additional comments or remarks regarding the fair distribution of authorship in research teams. This paper presents the analysis of these comments, categorized into four main themes: (1) disagreements, (2) questionable behavior, (3) external influences regarding authorship, and (4) values promoted by researchers. Results suggest that some respondents find ways to effectively manage disagreements in a collegial fashion. Conversely, others explain how distribution of authorship can become a "blood sport" or a "horror story" which can negatively affect researchers' wellbeing, scientific productivity and integrity. Researchers fear authorship discussions and often try to avoid openly discussing the situation which can strain team interactions. Unethical conduct is more likely to result from deceit, favoritism, and questionable mentorship and may become more egregious when there is constant bullying and discrimination. Although values of collegiality, transparency and fairness were promoted by researchers, rank and need for success often overpowered ethical decision-making. This research provides new insight into contextual specificities related to fair authorship distribution that can be instrumental in developing applicable training tools to identify, prevent, and mitigate authorship disagreement.
| S-EPMC6891155 | biostudies-literature
Project description:BackgroundAlthough improving, development of drugs and devices for children is still less effective than for adults. Pediatric academicians play an important role in the bench-to-bedside research process, but much remains to be done to improve their contributions.ObjectiveTo provide a non-comprehensive review of selected literature based on my own personal experience as a U.S. based academic researcher who has spent over 4 decades doing pediatric drug and device development.MethodsThis commentary presents a summary of a talk given at a recent pediatric drug development conference. The observations and conclusions reached were based on the author's (largely US) experience and review of past history, the role of academicians in this process, some successful models of public-private collaboration, available funding, and barriers that remain to be overcome.ResultsPediatric-specific legislation and more available funding have increased participation from and successes of US academicians in the pediatric drug and device development process. Incentive based public-private collaborations have been particularly successful. However, academicians still face both attitude and practical barriers to success.ConclusionsChanges are needed if academicians are to maximize their involvement in pediatric drug and device development.
| S-EPMC6677781 | biostudies-literature
Project description:For research domains such as life sciences, which pursue fundamental scientific understanding and applications intended for immediate use, academic entrepreneurship has played a pivotal role in commercialization. This paper presents an evaluation method of researchers related to user-inspired fundamental research, using global databases of startup finances and academic research papers of "startup readiness." Case studies of startups related to biopharmaceutical research topics suggest that the biopharmaceutical field has rich opportunities stemming from scientific research, commercialization, and entrepreneurship. This evaluation method sorts specific industry segments by which financing activities are active, and by which related growing research topics attract increased academic attention. We constructed networks of author citation and co-authorship from paper citation networks related to research topics in industry segments in the biopharmaceutical domain. Results obtained across all research topics we surveyed demonstrated that authors in the top 10% of degree centrality ranking in both networks are far more likely to be startup participants than other authors. Our computational approach might provide convenient, dynamic, global, and real-time understanding of the "startup readiness" of researchers working with research topics for which academic attention is emerging in actively financed biopharmaceutical fields.
| S-EPMC7300125 | biostudies-literature
Project description:Researchers' networks have been subject to active modeling and analysis. Earlier literature mostly focused on citation or co-authorship networks reconstructed from annotated scientific publication databases, which have several limitations. Recently, general-purpose web search engines have also been utilized to collect information about social networks. Here we reconstructed, using web search engines, a network representing the relatedness of researchers to their peers as well as to various research topics. Relatedness between researchers and research topics was characterized by visibility boost-increase of a researcher's visibility by focusing on a particular topic. It was observed that researchers who had high visibility boosts by the same research topic tended to be close to each other in their network. We calculated correlations between visibility boosts by research topics and researchers' interdisciplinarity at the individual level (diversity of topics related to the researcher) and at the social level (his/her centrality in the researchers' network). We found that visibility boosts by certain research topics were positively correlated with researchers' individual-level interdisciplinarity despite their negative correlations with the general popularity of researchers. It was also found that visibility boosts by network-related topics had positive correlations with researchers' social-level interdisciplinarity. Research topics' correlations with researchers' individual- and social-level interdisciplinarities were found to be nearly independent from each other. These findings suggest that the notion of "interdisciplinarity" of a researcher should be understood as a multi-dimensional concept that should be evaluated using multiple assessment means.
| S-EPMC3374816 | biostudies-other
Project description:Current discussions on improving the reproducibility of science often revolve around statistical innovations. However, equally important for improving methodological rigour is a valid operationalization of phenomena. Operationalization is the process of translating theoretical constructs into measurable laboratory quantities. Thus, the validity of operationalization is central for the quality of empirical studies. But do differences in the validity of operationalization affect the way scientists evaluate scientific literature? To investigate this, we manipulated the strength of operationalization of three published studies and sent them to researchers via email. In the first task, researchers were presented with a summary of the Method and Result section from one of the studies and were asked to guess the hypothesis that was investigated via a multiple-choice questionnaire. In a second task, researchers were asked to rate the perceived quality of the study. Our results show that (1) researchers are better at inferring the underlying research question from empirical results if the operationalization is more valid, but (2) the different validity is only to some extent reflected in a judgement of the study's quality. These results combined give partial corroboration to the notion that researchers' evaluations of research results are not affected by operationalization validity.
| S-EPMC8424321 | biostudies-literature