Dataset Information

Causal Datasheet for Datasets: An Evaluation Guide for Real-World Data Analysis and Data Collection Design Using Bayesian Networks.

ABSTRACT: Developing data-driven solutions that address real-world problems requires understanding of these problems' causes and how their interaction affects the outcome-often with only observational data. Causal Bayesian Networks (BN) have been proposed as a powerful method for discovering and representing the causal relationships from observational data as a Directed Acyclic Graph (DAG). BNs could be especially useful for research in global health in Lower and Middle Income Countries, where there is an increasing abundance of observational data that could be harnessed for policy making, program evaluation, and intervention design. However, BNs have not been widely adopted by global health professionals, and in real-world applications, confidence in the results of BNs generally remains inadequate. This is partially due to the inability to validate against some ground truth, as the true DAG is not available. This is especially problematic if a learned DAG conflicts with pre-existing domain doctrine. Here we conceptualize and demonstrate an idea of a "Causal Datasheet" that could approximate and document BN performance expectations for a given dataset, aiming to provide confidence and sample size requirements to practitioners. To generate results for such a Causal Datasheet, a tool was developed which can generate synthetic Bayesian networks and their associated synthetic datasets to mimic real-world datasets. The results given by well-known structure learning algorithms and a novel implementation of the OrderMCMC method using the Quotient Normalized Maximum Likelihood score were recorded. These results were used to populate the Causal Datasheet, and recommendations could be made dependent on whether expected performance met user-defined thresholds. We present our experience in the creation of Causal Datasheets to aid analysis decisions at different stages of the research process. First, one was deployed to help determine the appropriate sample size of a planned study of sexual and reproductive health in Madhya Pradesh, India. Second, a datasheet was created to estimate the performance of an existing maternal health survey we conducted in Uttar Pradesh, India. Third, we validated generated performance estimates and investigated current limitations on the well-known ALARM dataset. Our experience demonstrates the utility of the Causal Datasheet, which can help global health practitioners gain more confidence when applying BNs.

SUBMITTER: Butcher B

PROVIDER: S-EPMC8320747 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe combination of genotypic and genome-wide expression data arising from segregating populations offers an unprecedented opportunity to model and dissect complex phenotypes. The immense potential offered by these data derives from the fact that genotypic variation is the sole source of perturbation and can therefore be used to reconcile changes in gene expression programs with the parental genotypes. To date, several methodologies have been developed for modeling eQTL data. These methods generally leverage genotypic data to resolve causal relationships among gene pairs implicated as associates in the expression data. In particular, leading studies have augmented Bayesian networks with genotypic data, providing a powerful framework for learning and modeling causal relationships. While these initial efforts have provided promising results, one major drawback associated with these methods is that they are generally limited to resolving causal orderings for transcripts most proximal to the genomic loci. In this manuscript, we present a probabilistic method capable of learning the causal relationships between transcripts at all levels in the network. We use the information provided by our method as a prior for Bayesian network structure learning, resulting in enhanced performance for gene network reconstruction.ResultsUsing established protocols to synthesize eQTL networks and corresponding data, we show that our method achieves improved performance over existing leading methods. For the goal of gene network reconstruction, our method achieves improvements in recall ranging from 20% to 90% across a broad range of precision levels and for datasets of varying sample sizes. Additionally, we show that the learned networks can be utilized for expression quantitative trait loci mapping, resulting in upwards of 10-fold increases in recall over traditional univariate mapping.ConclusionsUsing the information from our method as a prior for Bayesian network structure learning yields large improvements in accuracy for the tasks of gene network reconstruction and expression quantitative trait loci mapping. In particular, our method is effective for establishing causal relationships between transcripts located both proximally and distally from genomic loci.

Project description:BackgroundElectronic medical records (EMRs) have the highest value among real-world data (RWD). The aim of the present study was to propose a data collection framework of EMR-based RWD to evaluate the effectiveness and safety of cancer drugs by conducting a nationwide real-world study based on the Korean Cancer Study Group.MethodsWe considered all patients who received ramucirumab plus paclitaxel (RAM/PTX) for gastric cancer and trastuzumab emtansine (T-DM1) for breast cancer at relevant institutions in South Korea. Standard operating procedures for systematic data collection were prospectively developed. Investigator reliability was evaluated using the concordance rate between the recommended input value for representative fictional cases and the input value of each investigator. Reliability of collected data was evaluated twice during the study period at three institutions randomly selected using the concordance rate between the previously collected data and data collected by an independent investigator. The reliability results of the investigators and collected data were used for revision of the electronic data capture system and site training.ResultsBetween the starting date of medical insurance coverage and December 2018, a total of 1063 patients at 56 institutions in the RAM/PTX cohort and 824 patients at 60 institutions in the T-DM1 cohort were included. Mean investigator reliability in the RAM/PTX and T-DM1 cohorts was 73.5% and 71.9%, respectively. Mean reliability of collected data in the RAM/PTX and T-DM1 cohort was 90.0% for both cohorts in the first analysis and 89.0% and 84.0% in the second analysis, respectively. Mean missing values of the RAM/PTX and T-DM1 cohorts at the time of simulation of fictional cases and final data analysis decreased from 20.7% to 0.46% and from 18.5% to 0.76%, respectively.ConclusionThis real-world study provides a framework that ensures relevance and reliability of EMR-based RWD for evaluating the effectiveness and safety of cancer drugs.

Dataset Information

Causal Datasheet for Datasets: An Evaluation Guide for Real-World Data Analysis and Data Collection Design Using Bayesian Networks.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets