Quantitative Bias Analysis for Collaborative Science.
Ontology highlight
ABSTRACT:
SUBMITTER: Weuve J
PROVIDER: S-EPMC6066441 | biostudies-literature | 2018 Sep
REPOSITORIES: biostudies-literature
Publications
Quantitative Bias Analysis for Collaborative Science.
Weuve Jennifer J Sagiv Sharon K SK Fox Matthew P MP
Epidemiology (Cambridge, Mass.) 20180901 5
Similar Datasets
Project description:Data-intensive research continues to expand with the goal of improving healthcare delivery, clinical decision-making, and patient outcomes. Quantitative scientists, such as biostatisticians, epidemiologists, and informaticists, are tasked with turning data into health knowledge. In academic health centres, quantitative scientists are critical to the missions of biomedical discovery and improvement of health. Many academic health centres have developed centralized Quantitative Science Units which foster dual goals of professional development of quantitative scientists and producing high quality, reproducible domain research. Such units then develop teams of quantitative scientists who can collaborate with researchers. However, existing literature does not provide guidance on how such teams are formed or how to manage and sustain them. Leaders of Quantitative Science Units across six institutions formed a working group to examine common practices and tools that can serve as best practices for Quantitative Science Units that wish to achieve these dual goals through building long-term partnerships with researchers. The results of this working group are presented to provide tools and guidance for Quantitative Science Units challenged with developing, managing, and evaluating Quantitative Science Teams. This guidance aims to help Quantitative Science Units effectively participate in and enhance the research that is conducted throughout the academic health centre-shaping their resources to fit evolving research needs.
| S-EPMC11340681 | biostudies-literature
Project description: Not available
| S-EPMC6284825 | biostudies-literature
Project description:Citizen science and automated collection methods increasingly depend on image recognition to provide the amounts of observational data research and management needs. Recognition models, meanwhile, also require large amounts of data from these sources, creating a feedback loop between the methods and tools. Species that are harder to recognize, both for humans and machine learning algorithms, are likely to be under-reported, and thus be less prevalent in the training data. As a result, the feedback loop may hamper training mostly for species that already pose the greatest challenge. In this study, we trained recognition models for various taxa, and found evidence for a 'recognizability bias', where species that are more readily identified by humans and recognition models alike are more prevalent in the available image data. This pattern is present across multiple taxa, and does not appear to relate to differences in picture quality, biological traits or data collection metrics other than recognizability. This has implications for the expected performance of future models trained with more data, including such challenging species.
| S-EPMC9890120 | biostudies-literature
Project description:Numerous biases are believed to affect the scientific literature, but their actual prevalence across disciplines is unknown. To gain a comprehensive picture of the potential imprint of bias in science, we probed for the most commonly postulated bias-related patterns and risk factors, in a large random sample of meta-analyses taken from all disciplines. The magnitude of these biases varied widely across fields and was overall relatively small. However, we consistently observed a significant risk of small, early, and highly cited studies to overestimate effects and of studies not published in peer-reviewed journals to underestimate them. We also found at least partial confirmation of previous evidence suggesting that US studies and early studies might report more extreme effects, although these effects were smaller and more heterogeneously distributed across meta-analyses and disciplines. Authors publishing at high rates and receiving many citations were, overall, not at greater risk of bias. However, effect sizes were likely to be overestimated by early-career researchers, those working in small or long-distance collaborations, and those responsible for scientific misconduct, supporting hypotheses that connect bias to situational factors, lack of mutual control, and individual integrity. Some of these patterns and risk factors might have modestly increased in intensity over time, particularly in the social sciences. Our findings suggest that, besides one being routinely cautious that published small, highly-cited, and earlier studies may yield inflated results, the feasibility and costs of interventions to attenuate biases in the literature might need to be discussed on a discipline-specific and topic-specific basis.
| S-EPMC5389310 | biostudies-literature
Project description:Background and objectivesSince the launch of Dublin City University's Age-Friendly University (AFU) Initiative in 2012, relatively little empirical research has been published on its feasibility or implementation by institutions of higher learning. This article describes how collaborative citizen science-a research method where professional researchers and community members work together across multiple stages of the research process (e.g., data collection, analysis, and/or knowledge mobilization) to investigate an issue-was used to identify barriers and supports to university age-friendliness at the University of Manitoba (UofM) in Canada.Research design and methodsTen citizen scientists each completed 1 data collection walk around the UofM campus and used a tablet application to document AFU barriers and supports via photographs and accompanying audio commentaries. The citizen scientists and university researchers then worked together in 2 analysis sessions to identify AFU priority areas and brainstorm recommendations for institutional change. These were then presented to a group of interested university stakeholders.ResultsThe citizen scientists collected 157 photos documenting AFU barriers and supports on campus. Accessibility, signage, and transportation were identified as being the most pressing issues for the university to address to improve overall age-friendliness.Discussion and implicationsWe suggest that academic institutions looking to complete assessments of their age-friendliness, particularly those exploring physical barriers and supports, could benefit from incorporating older citizen scientists into the process of collecting, analyzing, and mobilizing findings.
| S-EPMC8673440 | biostudies-literature
Project description:When informed by an understanding of cognitive science, radiologists' workstations could become collaborative to improve radiologists' performance and job satisfaction. The authors review relevant literature and present several promising areas of research, including image toggling, eye tracking, cognitive computing, intelligently restricted messaging, work habit tracking, and innovative input devices. The authors call for more research in "perceptual design," a promising field that can complement advances in computer-aided detection.
| S-EPMC5537723 | biostudies-literature
Project description:Globus, developed as Software-as-a-Service (SaaS) for research data management, also provides APIs that constitute a flexible and powerful Platform-as-a-Service (PaaS) to which developers can outsource data management activities such as transfer and sharing, as well as identity, profile and group management. By providing these frequently important but always challenging capabilities as a service, accessible over the network, Globus PaaS streamlines web application development and makes it easy for individuals, teams, and institutions to create collaborative applications such as science gateways for science communities. We introduce the capabilities of this platform and review representative applications.
| S-EPMC4309390 | biostudies-other
Project description:Quantitative bias analysis can be used to empirically assess how far study estimates are from the truth (i.e., an estimate that is free of bias). These methods can be used to explore the potential impact of confounding bias, selection bias (collider stratification bias), and information bias. Quantitative bias analysis includes methods that can be used to check the robustness of study findings to multiple types of bias and methods that use simulation studies to generate data and understand the hypothetical impact of specific types of bias in a simulated data set. In this article, we review 2 strategies for quantitative bias analysis: 1) traditional probabilistic quantitative bias analysis and 2) quantitative bias analysis with generated data. An important difference between the 2 strategies relates to the type of data (real vs. generated data) used in the analysis. Monte Carlo simulations are used in both approaches, but the simulation process is used for different purposes in each. For both approaches, we outline and describe the steps required to carry out the quantitative bias analysis and also present a bias-analysis tutorial demonstrating how both approaches can be applied in the context of an analysis for selection bias. Our goal is to highlight the utility of quantitative bias analysis for practicing epidemiologists and increase the use of these methods in the epidemiologic literature.
| S-EPMC9005059 | biostudies-literature
Project description:Although variables are often measured with error, the impact of measurement error on machine-learning predictions is seldom quantified. The purpose of this study was to assess the impact of measurement error on the performance of random-forest models and variable importance. First, we assessed the impact of misclassification (i.e., measurement error of categorical variables) of predictors on random-forest model performance (e.g., accuracy, sensitivity) and variable importance (mean decrease in accuracy) using data from the National Comorbidity Survey Replication (2001-2003). Second, we created simulated data sets in which we knew the true model performance and variable importance measures and could verify that quantitative bias analysis was recovering the truth in misclassified versions of the data sets. Our findings showed that measurement error in the data used to construct random forests can distort model performance and variable importance measures and that bias analysis can recover the correct results. This study highlights the utility of applying quantitative bias analysis in machine learning to quantify the impact of measurement error on study results.
| S-EPMC8408353 | biostudies-literature
Project description:A concerning amount of biomedical research is not reproducible. Unreliable results impede empirical progress in medical science, ultimately putting patients at risk. Many proximal causes of this irreproducibility have been identified, a major one being inappropriate statistical methods and analytical choices by investigators. Within this, we formally quantify the impact of inappropriate redaction beyond a threshold value in biomedical science. This is effectively truncation of a dataset by removing extreme data points, and we elucidate its potential to accidentally or deliberately engineer a spurious result in significance testing. We demonstrate that the removal of a surprisingly small number of data points can be used to dramatically alter a result. It is unknown how often redaction bias occurs in the broader literature, but given the risk of distortion to the literature involved, we suggest that it must be studiously avoided, and mitigated with approaches to counteract any potential malign effects to the research quality of medical science.
| S-EPMC8633797 | biostudies-literature