Dataset Information

Accuracy of automatic syndromic classification of coded emergency department diagnoses in identifying mental health-related presentations for public health surveillance.

ABSTRACT:

Background

Syndromic surveillance in emergency departments (EDs) may be used to deliver early warnings of increases in disease activity, to provide situational awareness during events of public health significance, to supplement other information on trends in acute disease and injury, and to support the development and monitoring of prevention or response strategies. Changes in mental health related ED presentations may be relevant to these goals, provided they can be identified accurately and efficiently. This study aimed to measure the accuracy of using diagnostic codes in electronic ED presentation records to identify mental health-related visits.

Methods

We selected a random sample of 500 records from a total of 1,815,588 ED electronic presentation records from 59 NSW public hospitals during 2010. ED diagnoses were recorded using any of ICD-9, ICD-10 or SNOMED CT classifications. Three clinicians, blinded to the automatically generated syndromic grouping and each other's classification, reviewed the triage notes and classified each of the 500 visits as mental health-related or not. A "mental health problem presentation" for the purposes of this study was defined as any ED presentation where either a mental disorder or a mental health problem was the reason for the ED visit. The combined clinicians' assessment of the records was used as reference standard to measure the sensitivity, specificity, and positive and negative predictive values of the automatic classification of coded emergency department diagnoses. Agreement between the reference standard and the automated coded classification was estimated using the Kappa statistic.

Results

Agreement between clinician's classification and automated coded classification was substantial (Kappa = 0.73. 95% CI: 0.58 - 0.87). The automatic syndromic grouping of coded ED diagnoses for mental health-related visits was found to be moderately sensitive (68% 95% CI: 46%-84%) and highly specific at 99% (95% CI: 98%-99.7%) when compared with the reference standard in identifying mental health related ED visits. Positive predictive value was 81% (95% CI: 0.57 - 0.94) and negative predictive value was 98% (95% CI: 0.97-0.99).

Conclusions

Mental health presentations identified using diagnoses coded with various classifications in electronic ED presentation records offers sufficient accuracy for application in near real-time syndromic surveillance.

SUBMITTER: Liljeqvist HT

PROVIDER: S-EPMC4177714 | biostudies-literature | 2014 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Accuracy of automatic syndromic classification of coded emergency department diagnoses in identifying mental health-related presentations for public health surveillance.

Liljeqvist Henning T G HT Muscatello David D Sara Grant G Dinh Michael M Lawrence Glenda L GL

BMC medical informatics and decision making 20140923

<h4>Background</h4>Syndromic surveillance in emergency departments (EDs) may be used to deliver early warnings of increases in disease activity, to provide situational awareness during events of public health significance, to supplement other information on trends in acute disease and injury, and to support the development and monitoring of prevention or response strategies. Changes in mental health related ED presentations may be relevant to these goals, provided they can be identified accurate ...[more]

PMID: 25245567

Similar Datasets

Project description:BackgroundSyndromic surveillance provides public health intelligence to aid in early warning and monitoring of public health impacts (e.g. seasonal influenza), or reassurance when an impact has not occurred. Using information collected during routine patient care, syndromic surveillance can be based on signs/symptoms/preliminary diagnoses. This approach makes syndromic surveillance much timelier than surveillance requiring laboratory confirmed diagnoses. The provision of healthcare services and patient access to them varies globally. However, emergency departments (EDs) exist worldwide, providing unscheduled urgent care to people in acute need. This provision of care makes ED syndromic surveillance (EDSyS) a potentially valuable tool for public health surveillance internationally. The objective of this study was to identify and describe the key characteristics of EDSyS systems that have been established and used globally.MethodsWe systematically reviewed studies published in peer review journals and presented at International Society of Infectious Disease Surveillance conferences (up to and including 2017) to identify EDSyS systems which have been created and used for public health purposes. Search criteria developed to identify "emergency department" and "syndromic surveillance" were applied to NICE healthcare, Global Health and Scopus databases.ResultsIn total, 559 studies were identified as eligible for inclusion in the review, comprising 136 journal articles and 423 conference abstracts/papers. From these studies we identified 115 EDSyS systems in 15 different countries/territories across North America, Europe, Asia and Australasia. Systems ranged from local surveillance based on a single ED, to comprehensive national systems. National EDSyS systems were identified in 8 countries/territories: 2 reported inclusion of ≥85% of ED visits nationally (France and Taiwan).ConclusionsEDSyS provides a valuable tool for the identification and monitoring of trends in severe illness. Technological advances, particularly in the emergency care patient record, have enabled the evolution of EDSyS over time. EDSyS reporting has become closer to 'real-time', with automated, secure electronic extraction and analysis possible on a daily, or more frequent basis. The dissemination of methods employed and evidence of successful application to public health practice should be encouraged to support learning from best practice, enabling future improvement, harmonisation and collaboration between systems in future.Prospero numberCRD42017069150 .

Project description:The science of surveillance is rapidly evolving due to changes in public health information and preparedness as national security issues, new information technologies and health reform. As the Emergency Department has become a much more utilized venue for acute care, it has also become a more attractive data source for disease surveillance. In recent years, influenza surveillance from the Emergency Department has increased in scope and breadth and has resulted in innovative and increasingly accepted methods of surveillance for influenza and influenza-like-illness (ILI). We undertook a systematic review of published Emergency Department-based influenza and ILI syndromic surveillance systems. A PubMed search using the keywords "syndromic", "surveillance", "influenza" and "emergency" was performed. Manuscripts were included in the analysis if they described (1) data from an Emergency Department (2) surveillance of influenza or ILI and (3) syndromic or clinical data. Meeting abstracts were excluded. The references of included manuscripts were examined for additional studies. A total of 38 manuscripts met the inclusion criteria, describing 24 discrete syndromic surveillance systems. Emergency Department-based influenza syndromic surveillance has been described worldwide. A wide variety of clinical data was used for surveillance, including chief complaint/presentation, preliminary or discharge diagnosis, free text analysis of the entire medical record, Google flu trends, calls to teletriage and help lines, ambulance dispatch calls, case reports of H1N1 in the media, markers of ED crowding, admission and Left Without Being Seen rates. Syndromes used to capture influenza rates were nearly always related to ILI (i.e. fever +/- a respiratory or constitutional complaint), however, other syndromes used for surveillance included fever alone, "respiratory complaint" and seizure. Two very large surveillance networks, the North American DiSTRIBuTE network and the European Triple S system have collected large-scale Emergency Department-based influenza and ILI syndromic surveillance data. Syndromic surveillance for influenza and ILI from the Emergency Department is becoming more prevalent as a measure of yearly influenza outbreaks.

Project description:Syndromic surveillance systems are plagued by high false-positive rates. In chronic disease monitoring, investigators have identified several factors that predict the accuracy of case definitions based on diagnoses in administrative data, and some have even incorporated these predictors into novel case detection methods, resulting in a significant improvement in case definition accuracy. Based on findings from these studies, we sought to identify physician, patient, encounter, and billing characteristics associated with the positive predictive value (PPV) of case definitions for 5 syndromes (fever, gastrointestinal, neurological, rash, and respiratory (including influenza-like illness)).The study sample comprised 4,330 syndrome-positive visits from the claims of 1,098 randomly-selected physicians working in Quebec, Canada in 2005-2007. For each visit, physician-facilitated chart review was used to assess whether the same syndrome was present in the medical chart (gold standard). We used multivariate logistic regression analyses to estimate the association between claim-chart agreement about the presence of a syndrome and physician, patient, encounter, and billing characteristics.The likelihood of the medical chart agreeing with the physician claim about the presence of a syndrome was higher when the treating physician had billed many visits for the same syndrome recently (ORper 10 visit, 1.05; 95% CI, 1.01-1.08), had a lower workload (ORper 10 claims, 0.93; 95% CI, 0.90-0.97), and when the patient was younger (ORper 5 years of age, 0.96; 95% CI, 0.94-0.97), and less socially deprived (ORmost versus least deprived, 0.76; 95% CI, 0.60-0.95).Many physician, patient, encounter, and billing characteristics associated with the PPV of surveillance case definition are accessible to public health, and could be used to reduce false-positive alerts by surveillance systems, either by focusing on the data most likely to be accurate, or by adjusting the observed data for known biases in diagnosis reporting and performing surveillance using the adjusted values.

Project description:BackgroundAutomated Emergency Department syndromic surveillance systems (ED-SyS) are useful tools in routine surveillance activities and during mass gathering events to rapidly detect public health threats. To improve the existing surveillance infrastructure in a lower-resourced rural/remote setting and enhance monitoring during an upcoming mass gathering event, an automated low-cost and low-resources ED-SyS was developed and validated in Yukon, Canada.MethodsSyndromes of interest were identified in consultation with the local public health authorities. For each syndrome, case definitions were developed using published resources and expert elicitation. Natural language processing algorithms were then written using Stata LP 15.1 (Texas, USA) to detect syndromic cases from three different fields (e.g., triage notes; chief complaint; discharge diagnosis), comprising of free-text and standardized codes. Validation was conducted using data from 19,082 visits between October 1, 2018 to April 30, 2019. The National Ambulatory Care Reporting System (NACRS) records were used as a reference for the inclusion of International Classification of Disease, 10th edition (ICD-10) diagnosis codes. The automatic identification of cases was then manually validated by two raters and results were used to calculate positive predicted values for each syndrome and identify improvements to the detection algorithms.ResultsA daily secure file transfer of Yukon's Meditech ED-Tracker system data and an aberration detection plan was set up. A total of six syndromes were originally identified for the syndromic surveillance system (e.g., Gastrointestinal, Influenza-like-Illness, Mumps, Neurological Infections, Rash, Respiratory), with an additional syndrome added to assist in detecting potential cases of COVID-19. The positive predictive value for the automated detection of each syndrome ranged from 48.8-89.5% to 62.5-94.1% after implementing improvements identified during validation. As expected, no records were flagged for COVID-19 from our validation dataset.ConclusionsThe development and validation of automated ED-SyS in lower-resourced settings can be achieved without sophisticated platforms, intensive resources, time or costs. Validation is an important step for measuring the accuracy of syndromic surveillance, and ensuring it performs adequately in a local context. The use of three different fields and integration of both free-text and structured fields improved case detection.

Project description:ContextAs the science and practice of syndromic surveillance (SyS) evolve, it has increasing utility for public health surveillance at the local level. Local health departments (LHDs) require specific organizational and workforce capabilities to use SyS data. In 2013, more than half of the LHDs reported using SyS, although little has been reported about LHD workforce capabilities in SyS.ObjectiveTo conduct an assessment of self-reported knowledge and skills in SyS tasks to effectively target technical assistance to different levels of LHD need.Design, setting, and participantsA stratified sampling design based on LHD jurisdiction population and SyS status was employed. Data were drawn from the 2015 Biosurveillance Needs Assessment Survey, which captured variables related to LHD use of SyS, management of systems, and self-reported proficiencies in a typology of SyS functionalities developed by a workgroup of subject matter experts in SyS. Respondents were US-based LHD public health practitioners. Estimation weights were applied during analysis to determine the national representation of the responses.Main outcome measuresRespondents self-reported proficiency in 26 SyS tasks within 5 categories, analyzed by LHD jurisdiction size and respondents' years of SyS experience.ResultsSyS expertise varied widely across LHDs. Less than 50% of workers who have access to SyS demonstrated overall proficiency within any of the task areas: communication, data use, data analysis, quality monitoring and assurance, and system design and development. SyS users were strongest in data use tasks. Proficiency in SyS practice corresponded directly with respondents' years of SyS experience and the LHD jurisdiction size.ConclusionSyS practitioners display a wide range of proficiencies both within and across SyS tasks. Considerable gaps in proficiencies of all areas of SyS practice indicate a need for technical assistance and knowledge dissemination to improve SyS practice as an important component of an LHD surveillance strategy.

Dataset Information

Accuracy of automatic syndromic classification of coded emergency department diagnoses in identifying mental health-related presentations for public health surveillance.

Background

Methods

Results

Conclusions

Publications

Accuracy of automatic syndromic classification of coded emergency department diagnoses in identifying mental health-related presentations for public health surveillance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets