Dataset Information

Using Bayes' rule to define the value of evidence from syndromic surveillance.

ABSTRACT: In this work we propose the adoption of a statistical framework used in the evaluation of forensic evidence as a tool for evaluating and presenting circumstantial "evidence" of a disease outbreak from syndromic surveillance. The basic idea is to exploit the predicted distributions of reported cases to calculate the ratio of the likelihood of observing n cases given an ongoing outbreak over the likelihood of observing n cases given no outbreak. The likelihood ratio defines the Value of Evidence (V). Using Bayes' rule, the prior odds for an ongoing outbreak are multiplied by V to obtain the posterior odds. This approach was applied to time series on the number of horses showing clinical respiratory symptoms or neurological symptoms. The separation between prior beliefs about the probability of an outbreak and the strength of evidence from syndromic surveillance offers a transparent reasoning process suitable for supporting decision makers. The value of evidence can be translated into a verbal statement, as often done in forensics or used for the production of risk maps. Furthermore, a Bayesian approach offers seamless integration of data from syndromic surveillance with results from predictive modeling and with information from other sources such as disease introduction risk assessments.

SUBMITTER: Andersson MG

PROVIDER: S-EPMC4218722 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using Bayes' rule to define the value of evidence from syndromic surveillance.

Andersson Mats Gunnar MG Faverjon Céline C Vial Flavie F Legrand Loïc L Leblond Agnès A

PloS one 20141103 11

In this work we propose the adoption of a statistical framework used in the evaluation of forensic evidence as a tool for evaluating and presenting circumstantial "evidence" of a disease outbreak from syndromic surveillance. The basic idea is to exploit the predicted distributions of reported cases to calculate the ratio of the likelihood of observing n cases given an ongoing outbreak over the likelihood of observing n cases given no outbreak. The likelihood ratio defines the Value of Evidence ( ...[more]

PMID: 25364823

Similar Datasets

Project description:MotivationIntrinsically disordered regions of proteins play an essential role in the regulation of various biological processes. Key to their regulatory function is often the binding to globular protein domains via sequence elements known as molecular recognition features (MoRFs). Development of computational tools for the identification of candidate MoRF locations in amino acid sequences is an important task and an area of growing interest. Given the relative sparseness of MoRFs in protein sequences, the accuracy of the available MoRF predictors is often inadequate for practical usage, which leaves a significant need and room for improvement. In this work, we introduce MoRFCHiBi_Web, which predicts MoRF locations in protein sequences with higher accuracy compared to current MoRF predictors.MethodsThree distinct and largely independent property scores are computed with component predictors and then combined to generate the final MoRF propensity scores. The first score reflects the likelihood of sequence windows to harbour MoRFs and is based on amino acid composition and sequence similarity information. It is generated by MoRFCHiBi using small windows of up to 40 residues in size. The second score identifies long stretches of protein disorder and is generated by ESpritz with the DisProt option. Lastly, the third score reflects residue conservation and is assembled from PSSM files generated by PSI-BLAST. These propensity scores are processed and then hierarchically combined using Bayes rule to generate the final MoRFCHiBi_Web predictions.ResultsMoRFCHiBi_Web was tested on three datasets. Results show that MoRFCHiBi_Web outperforms previously developed predictors by generating less than half the false positive rate for the same true positive rate at practical threshold values. This level of accuracy paired with its relatively high processing speed makes MoRFCHiBi_Web a practical tool for MoRF prediction.Availabilityhttp://morf.chibi.ubc.ca:8080/morf/.

Project description:IntroductionThe evaluation of peer-reviewed cases for error is key to quality assurance (QA) in emergency medicine, but defining error to ensure reviewer agreement and reproducibility remains elusive. The objective of this study was to create a consensus-based set of rules to systematically identify medical errors.MethodsThis is a prospective, observational study of all cases presented for peer review at an urban, tertiary care, academic medical center emergency department (ED) quality assurance (QA) committee between October 13, 2015, and September 14, 2016. Our hospital uses an electronic system enabling staff to self-identify QA issues for subsequent review. In addition, physician or patient complaints, 72-hour returns with admission, death within 24 hours, floor transfers to ICU < 24 hours, and morbidity and mortality conference cases are automatic triggers for review. Trained reviewers not involved in the patient's care use a structured 8-point Likert scale to assess for error and preventable or non-preventable adverse events. Cases where reviewers perceived a need for additional treatment, or that caused patient harm, are referred to a 20-member committee of emergency department leadership, attendings, residents, and nurses for consensus review. For this study, "rules" were proposed by the reviewers identifying the error and validated by consensus during each meeting. The committee then decided if a rule had been broken (error) or not broken (judgment call). If an error could not be phrased in terms of a rule broken, then it would not be considered an error. The rules were then evaluated by 2 reviewers and organized by theme into categories to determine common errors in emergency medicine.ResultsWe identified 108 episodes of rules broken in 103 cases within a database of 920 QA reviewed cases. In cases where a rule was broken and therefore an error was scored, the following 5 major themes emerged: (1) not acquiring necessary information (eg, not completing a relevant physical exam), N = 33 (31%); (2) not acting on data that were acquired (eg, abnormal vital signs or labs), N = 25 (23%); (3) knowledge gaps by clinicians (eg, not knowing to reduce a hernia), N = 16 (15%); (4) communication gaps (eg, discharge instructions), N = 17 (16%); and (5) systems issues (eg, improper patient registration), N = 17 (16%).ConclusionThe development of consensus-based rules may result in a more standardized and practical definition of error in emergency medicine to be used as a QA tool and a basis for research. The most common type of rule broken was not acquiring necessary information. A rule-based definition of medical error in emergency medicine may identify key areas for risk reduction strategies, help standardize medical QA, and improve patient care and physician education.

Project description:INTRODUCTION:The Risk Identification Unit (RIU) of the US Dept. of Agriculture's Center for Epidemiology and Animal Health (CEAH) conducts weekly surveillance of national livestock health data and routine coordination with agricultural stakeholders. As part of an initiative to increase the number of species, health issues, and data sources monitored, CEAH epidemiologists are building a surveillance system based on weekly syndromic counts of laboratory test orders in consultation with Colorado State University laboratorians and statistical analysts from the Johns Hopkins University Applied Physics Laboratory. Initial efforts focused on 12 years of equine test records from three state labs. Trial syndrome groups were formed based on RIU experience and published literature. Exploratory analysis, stakeholder input, and laboratory workflow details were needed to modify these groups and filter the corresponding data to eliminate alerting bias. Customized statistical detection methods were sought for effective monitoring based on specialized laboratory information characteristics and on the likely presentation and animal health significance of diseases associated with each syndrome. METHODS:Data transformation and syndrome formation focused on test battery type, test name, submitter source organization, and specimen type. We analyzed time series of weekly counts of tests included in candidate syndrome groups and conducted an iterative process of data analysis and veterinary consultation for syndrome refinement and record filters. This process produced a rule set in which records were directly classified into syndromes using only test name when possible, and otherwise, the specimen type or related body system was used with test name to determine the syndrome. Test orders associated with government regulatory programs, veterinary teaching hospital testing protocols, or research projects, rather than clinical concerns, were excluded. We constructed a testbed for sets of 1000 statistical trials and applied a stochastic injection process assuming lognormally distributed incubation periods to choose an alerting algorithm with the syndrome-required sensitivity and an alert rate within the specified acceptable range for each resulting syndrome. Alerting performance of the EARS C3 algorithm traditionally used by CEAH was compared to modified C2, CuSUM, and EWMA methods, with and without outlier removal and adjustments for the total weekly number of non-mandatory tests. RESULTS:The equine syndrome groups adopted for monitoring were abortion/reproductive, diarrhea/GI, necropsy, neurological, respiratory, systemic fungal, and tickborne. Data scales, seasonality, and variance differed widely among the weekly time series. Removal of mandatory and regulatory tests reduced weekly observed counts significantly-by >80% for diarrhea/GI syndrome. The RIU group studied outcomes associated with each syndrome and called for detection of single-week signals for most syndromes with expected false-alert intervals >8 and <52 weeks, 8-week signals for neurological and tickborne monitoring (requiring enhanced sensitivity), 6-week signals for respiratory, and 4-week signals for systemic fungal. From the test-bed trials, recommended methods, settings and thresholds were derived. CONCLUSIONS:Understanding of laboratory submission sources, laboratory workflow, and of syndrome-related outcomes are crucial to form syndrome groups for routine monitoring without artifactual alerting. Choices of methods, parameters, and thresholds varied by syndrome and depended strongly on veterinary epidemiologist-specified performance requirements.

Dataset Information

Using Bayes' rule to define the value of evidence from syndromic surveillance.

Publications

Using Bayes' rule to define the value of evidence from syndromic surveillance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets