Dataset Information

Mapping the discursive dimensions of the reproducibility crisis: A mixed methods analysis.

ABSTRACT: To those involved in discussions about rigor, reproducibility, and replication in science, conversation about the "reproducibility crisis" appear ill-structured. Seemingly very different issues concerning the purity of reagents, accessibility of computational code, or misaligned incentives in academic research writ large are all collected up under this label. Prior work has attempted to address this problem by creating analytical definitions of reproducibility. We take a novel empirical, mixed methods approach to understanding variation in reproducibility discussions, using a combination of grounded theory and correspondence analysis to examine how a variety of authors narrate the story of the reproducibility crisis. Contrary to expectations, this analysis demonstrates that there is a clear thematic core to reproducibility discussions, centered on the incentive structure of science, the transparency of methods and data, and the need to reform academic publishing. However, we also identify three clusters of discussion that are distinct from the main body of articles: one focused on reagents, another on statistical methods, and a final cluster focused on the heterogeneity of the natural world. Although there are discursive differences between scientific and popular articles, we find no strong differences in how scientists and journalists write about the reproducibility crisis. Our findings demonstrate the value of using qualitative methods to identify the bounds and features of reproducibility discourse, and identify distinct vocabularies and constituencies that reformers should engage with to promote change.

SUBMITTER: Nelson NC

PROVIDER: S-EPMC8270481 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe COVID-19 pandemic was accompanied by the accelerated uptake of virtual care, leading to a proliferation of virtual ward models as alternatives to facility-based care. Early in the pandemic, our program implemented a virtual mental health crisis ward (vWard) to provide options for individuals requiring intense psychiatric and/or crisis support but who preferred to remain in the community and were deemed safe to do so.ObjectiveThe aim of this study was to identify early learnings from the vWard, which was implemented rapidly in a resource-constrained environment, to inform the future state should it be sustained beyond the pandemic.MethodsMixed methods of data collection were used to evaluate provider perspectives on the vWard, develop archetypes for individuals who are a good fit for the vWard model, and create a driver diagram. Data sources included an anonymous survey of clinical and managerial staff involved in the vWard, a service planning workshop, and program discharge forms for all individuals admitted between March 2020 and April 2021. Survey responses were coded for themes under categories of "benefits" and "challenges." Discharge forms where the team indicated that the vWard was a good fit for an individual were examined for characteristics common to these admissions. These findings were reviewed in the service planning workshop and refined with input from the participants into patient archetypes. A driver diagram was created for the future state.ResultsSurvey respondents (N=60) represented diverse roles in crisis services and the vWard team. Ten providers took part in the service planning workshop. A total of 467 discharge forms were reviewed. The vWard was felt to be a model that worked by 39 survey respondents, one respondent felt it did not work, and the remaining participants had no response. Several benefits for the individual and the system were identified alongside challenges, including certain processes and materials related to the nature of rapid implementation during the pandemic, and others due to lack of fit for certain individuals. The model was felt to be a good fit for 67.5% of admissions. Four patient archetypes representing a good fit with the model were developed. The driver diagram connected the program aim with primary drivers of (1) reduce barriers to care; (2) improve outcomes; and (3) provide collaborative, patient- and family-centered care to secondary drivers and interventions that leveraged virtual technology among other crisis care interventions.ConclusionsDespite some challenges, the vWard demonstrated high levels of provider acceptance and a range of mechanisms by which the model works for a variety of patient archetypes. These early learnings provide a foundation for growth, sustainability, and spread of this model going forward beyond the pandemic.

Project description:We compared the repeatability, reproducibility (intra- and inter-measurer similarity), separative power and subjectivity (measurer effect on results) of four morphometric methods frequently used in ichthyological research, the "traditional" caliper-based (TRA) and truss-network (TRU) distance methods and two geometric methods that compare landmark coordinates on the body (GMB) and scales (GMS). In each case, measurements were performed three times by three measurers on the same specimen of three common cyprinid species (roach Rutilus rutilus (Linnaeus, 1758), bleak Alburnus alburnus (Linnaeus, 1758) and Prussian carp Carassius gibelio (Bloch, 1782)) collected from three closely-situated sites in the Lake Balaton catchment (Hungary) in 2014. TRA measurements were made on conserved specimens using a digital caliper, while TRU, GMB and GMS measurements were undertaken on digital images of the bodies and scales. In most cases, intra-measurer repeatability was similar. While all four methods were able to differentiate the source populations, significant differences were observed in their repeatability, reproducibility and subjectivity. GMB displayed highest overall repeatability and reproducibility and was least burdened by measurer effect. While GMS showed similar repeatability to GMB when fish scales had a characteristic shape, it showed significantly lower reproducability (compared with its repeatability) for each species than the other methods. TRU showed similar repeatability as the GMS. TRA was the least applicable method as measurements were obtained from the fish itself, resulting in poor repeatability and reproducibility. Although all four methods showed some degree of subjectivity, TRA was the only method where population-level detachment was entirely overwritten by measurer effect. Based on these results, we recommend a) avoidance of aggregating different measurer's datasets when using TRA and GMS methods; and b) use of image-based methods for morphometric surveys. Automation of the morphometric workflow would also reduce any measurer effect and eliminate measurement and data-input errors.

Project description:BackgroundThe study of safety culture and its relationship to patient care have been challenged by variation in definition, dimensionality and methods of assessment. This systematic review aimed to map methods to assess safety culture in hospitals, analyse the prevalence of these methods in the published research literature and examine the dimensions of safety culture captured through these processes.MethodsWe included studies reporting on quantitative, qualitative and mixed methods to assess safety culture in hospitals. The review was conducted using four academic databases (PubMed, CINAHL, Scopus and Web of Science) with studies from January 2008 to May 2020. A formal quality appraisal was not conducted. Study purpose, type of method and safety culture dimensions were extracted from all studies, coded thematically, and summarised narratively and using descriptive statistics where appropriate.ResultsA total of 694 studies were included. A third (n=244, 35.2%) had a descriptive or exploratory purpose, 225 (32.4%) tested relationships among variables, 129 (18.6%) evaluated an intervention, while 13.8% (n=96) had a methodological focus. Most studies exclusively used surveys (n=663; 95.5%), with 88 different surveys identified. Only 31 studies (4.5%) used qualitative or mixed methods. Thematic analysis identified 11 themes related to safety culture dimensions across the methods, with 'Leadership' being the most common. Qualitative and mixed methods approaches were more likely to identify additional dimensions of safety culture not covered by the 11 themes, including improvisation and contextual pressures.DiscussionWe assessed the extent to which safety culture dimensions mapped to specific quantitative and qualitative tools and methods of assessing safety culture. No single method or tool appeared to measure all 11 themes of safety culture. Risk of publication bias was high in this review. Future attempts to assess safety culture in hospitals should consider incorporating qualitative methods into survey studies to evaluate this multi-faceted construct.

Dataset Information

Mapping the discursive dimensions of the reproducibility crisis: A mixed methods analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets