Dataset Information

Diagnostic performance of electronic syndromic surveillance systems in acute care: a systematic review.

ABSTRACT:

Context

Healthcare Electronic Syndromic Surveillance (ESS) is the systematic collection, analysis and interpretation of ongoing clinical data with subsequent dissemination of results, which aid clinical decision-making.

Objective

To evaluate, classify and analyze the diagnostic performance, strengths and limitations of existing acute care ESS systems.

Data sources

All available to us studies in Ovid MEDLINE, Ovid EMBASE, CINAHL and Scopus databases, from as early as January 1972 through the first week of September 2012.

Study selection

Prospective and retrospective trials, examining the diagnostic performance of inpatient ESS and providing objective diagnostic data including sensitivity, specificity, positive and negative predictive values.

Data extraction

Two independent reviewers extracted diagnostic performance data on ESS systems, including clinical area, number of decision points, sensitivity and specificity. Positive and negative likelihood ratios were calculated for each healthcare ESS system. A likelihood matrix summarizing the various ESS systems performance was created.

Results

The described search strategy yielded 1639 articles. Of these, 1497 were excluded on abstract information. After full text review, abstraction and arbitration with a third reviewer, 33 studies met inclusion criteria, reporting 102,611 ESS decision points. The yielded I2 was high (98.8%), precluding meta-analysis. Performance was variable, with sensitivities ranging from 21% -100% and specificities ranging from 5%-100%.

Conclusions

There is significant heterogeneity in the diagnostic performance of the available ESS implements in acute care, stemming from the wide spectrum of different clinical entities and ESS systems. Based on the results, we introduce a conceptual framework using a likelihood ratio matrix for evaluation and meaningful application of future, frontline clinical decision support systems.

SUBMITTER: Kashiouris M

PROVIDER: S-EPMC3716308 | biostudies-literature | 2013

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Diagnostic performance of electronic syndromic surveillance systems in acute care: a systematic review.

Kashiouris M M O'Horo J C JC Pickering B W BW Herasevich V V

Applied clinical informatics 20130508 2

<h4>Context</h4>Healthcare Electronic Syndromic Surveillance (ESS) is the systematic collection, analysis and interpretation of ongoing clinical data with subsequent dissemination of results, which aid clinical decision-making.<h4>Objective</h4>To evaluate, classify and analyze the diagnostic performance, strengths and limitations of existing acute care ESS systems.<h4>Data sources</h4>All available to us studies in Ovid MEDLINE, Ovid EMBASE, CINAHL and Scopus databases, from as early as January ...[more]

PMID: 23874359

Similar Datasets

Project description:BackgroundSyndromic surveillance provides public health intelligence to aid in early warning and monitoring of public health impacts (e.g. seasonal influenza), or reassurance when an impact has not occurred. Using information collected during routine patient care, syndromic surveillance can be based on signs/symptoms/preliminary diagnoses. This approach makes syndromic surveillance much timelier than surveillance requiring laboratory confirmed diagnoses. The provision of healthcare services and patient access to them varies globally. However, emergency departments (EDs) exist worldwide, providing unscheduled urgent care to people in acute need. This provision of care makes ED syndromic surveillance (EDSyS) a potentially valuable tool for public health surveillance internationally. The objective of this study was to identify and describe the key characteristics of EDSyS systems that have been established and used globally.MethodsWe systematically reviewed studies published in peer review journals and presented at International Society of Infectious Disease Surveillance conferences (up to and including 2017) to identify EDSyS systems which have been created and used for public health purposes. Search criteria developed to identify "emergency department" and "syndromic surveillance" were applied to NICE healthcare, Global Health and Scopus databases.ResultsIn total, 559 studies were identified as eligible for inclusion in the review, comprising 136 journal articles and 423 conference abstracts/papers. From these studies we identified 115 EDSyS systems in 15 different countries/territories across North America, Europe, Asia and Australasia. Systems ranged from local surveillance based on a single ED, to comprehensive national systems. National EDSyS systems were identified in 8 countries/territories: 2 reported inclusion of ≥85% of ED visits nationally (France and Taiwan).ConclusionsEDSyS provides a valuable tool for the identification and monitoring of trends in severe illness. Technological advances, particularly in the emergency care patient record, have enabled the evolution of EDSyS over time. EDSyS reporting has become closer to 'real-time', with automated, secure electronic extraction and analysis possible on a daily, or more frequent basis. The dissemination of methods employed and evidence of successful application to public health practice should be encouraged to support learning from best practice, enabling future improvement, harmonisation and collaboration between systems in future.Prospero numberCRD42017069150 .

Project description:BackgroundTechnology-based systems can facilitate remote decision-making to triage patients to the appropriate level of care. Despite technologic advances, the effects of implementation of these systems on patient and utilization outcomes are unclear. We evaluated the effects of remote triage systems on healthcare utilization, case resolution, and patient safety outcomes.MethodsEnglish-language searches of MEDLINE (via PubMed), EMBASE, and CINAHL were performed from inception until July 2018. Randomized and nonrandomized comparative studies of remote triage services that reported healthcare utilization, case resolution, and patient safety outcomes were included. Two reviewers assessed study and intervention characteristics independently for study quality, strength of evidence, and risk of bias.ResultsThe literature search identified 5026 articles, of which eight met eligibility criteria. Five randomized, two controlled before-and-after, and one interrupted time series study assessed 3 categories of remote triage services: mode of delivery, triage professional type, and system organizational level. No study evaluated any other delivery mode other than telephone and in-person. Meta-analyses were unable to be performed because of study design and outcome heterogeneity; therefore, we narratively synthesized data. Overall, most studies did not demonstrate a decrease in primary care (PC) or emergency department (ED) utilization, with some studies showing a significant increase. Evidence suggested local, practice-based triage systems have greater case resolution and refer fewer patients to PC or ED services than regional/national systems. No study identified statistically significant differences in safety outcomes.ConclusionOur review found limited evidence that remote triage reduces the burden of PC or ED utilization. However, remote triage by telephone can produce a high rate of call resolution and appears to be safe. Further study of other remote triage modalities is needed to realize the promise of remote triage services in optimizing healthcare outcomes.Protocol registrationThis study was registered and followed a published protocol (PROSPERO: CRD42019112262).

Project description:The science of surveillance is rapidly evolving due to changes in public health information and preparedness as national security issues, new information technologies and health reform. As the Emergency Department has become a much more utilized venue for acute care, it has also become a more attractive data source for disease surveillance. In recent years, influenza surveillance from the Emergency Department has increased in scope and breadth and has resulted in innovative and increasingly accepted methods of surveillance for influenza and influenza-like-illness (ILI). We undertook a systematic review of published Emergency Department-based influenza and ILI syndromic surveillance systems. A PubMed search using the keywords "syndromic", "surveillance", "influenza" and "emergency" was performed. Manuscripts were included in the analysis if they described (1) data from an Emergency Department (2) surveillance of influenza or ILI and (3) syndromic or clinical data. Meeting abstracts were excluded. The references of included manuscripts were examined for additional studies. A total of 38 manuscripts met the inclusion criteria, describing 24 discrete syndromic surveillance systems. Emergency Department-based influenza syndromic surveillance has been described worldwide. A wide variety of clinical data was used for surveillance, including chief complaint/presentation, preliminary or discharge diagnosis, free text analysis of the entire medical record, Google flu trends, calls to teletriage and help lines, ambulance dispatch calls, case reports of H1N1 in the media, markers of ED crowding, admission and Left Without Being Seen rates. Syndromes used to capture influenza rates were nearly always related to ILI (i.e. fever +/- a respiratory or constitutional complaint), however, other syndromes used for surveillance included fever alone, "respiratory complaint" and seizure. Two very large surveillance networks, the North American DiSTRIBuTE network and the European Triple S system have collected large-scale Emergency Department-based influenza and ILI syndromic surveillance data. Syndromic surveillance for influenza and ILI from the Emergency Department is becoming more prevalent as a measure of yearly influenza outbreaks.

Project description:BackgroundElectronic guideline-based decision support systems have been suggested to successfully deliver the knowledge embedded in clinical practice guidelines. A number of studies have already shown positive findings for decision support systems such as drug-dosing systems and computer-generated reminder systems for preventive care services.MethodsA systematic literature search (1990 to December 2008) of the English literature indexed in the Medline database, Embase, the Cochrane Central Register of Controlled Trials, and CRD (DARE, HTA and NHS EED databases) was conducted to identify evaluation studies of electronic multi-step guideline implementation systems in ambulatory care settings. Important inclusion criterions were the multidimensionality of the guideline (the guideline needed to consist of several aspects or steps) and real-time interaction with the system during consultation. Clinical decision support systems such as one-time reminders for preventive care for which positive findings were shown in earlier reviews were excluded. Two comparisons were considered: electronic multidimensional guidelines versus usual care (comparison one) and electronic multidimensional guidelines versus other guideline implementation methods (comparison two).ResultsTwenty-seven publications were selected for analysis in this systematic review. Most designs were cluster randomized controlled trials investigating process outcomes more than patient outcomes. With success defined as at least 50% of the outcome variables being significant, none of the studies were successful in improving patient outcomes. Only seven of seventeen studies that investigated process outcomes showed improvements in process of care variables compared with the usual care group (comparison one). No incremental effect of the electronic implementation over the distribution of paper versions of the guideline was found, neither for the patient outcomes nor for the process outcomes (comparison two).ConclusionsThere is little evidence at the moment for the effectiveness of an increasingly used and commercialized instrument such as electronic multidimensional guidelines. After more than a decade of development of numerous electronic systems, research on the most effective implementation strategy for this kind of guideline-based decision support systems is still lacking. This conclusion implies a considerable risk towards inappropriate investments in ineffective implementation interventions and in suboptimal care.

Project description:BackgroundAn updated overview of ultrasound (US) for diagnosis of acute cholecystitis (AC) remains lacking. This systematic review was conducted to evaluate the diagnostic performance of US for AC.MethodsA systematic review was conducted following PRISMA guidelines. We meticulously screened articles from MEDLINE, Embase, and the Cochrane Library, spanning from inception to August 2023. We employed the search strategy combining the keywords "bedside US", "emergency US" or "point-of-care US" with "AC". Two reviewers independently screened the titles and abstracts of the retrieved articles to identify suitable studies. The inclusion criteria encompassed articles investigating the diagnostic performance of US for AC. Data regarding diagnostic performance, sonographers, and sonographic findings including the presence of gallstone, gallbladder (GB) wall thickness, peri-GB fluid, or sonographic Murphy sign were extracted, and a meta-analysis was executed. Case reports, editorials, and review articles were excluded, as well as studies focused on acalculous cholecystitis. The study quality was assessed with the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool.ResultsForty studies with 8,652 patients were included. The majority of studies had a low risk of bias and applicability concerns. US had a pooled sensitivity of 71% (95% CI, 69-72%), a specificity of 85% (95% CI, 84-86%), and an accuracy of 0.83 (95% CI, 0.82-0.83) for the diagnosis of AC. The pooled sensitivity and specificity were 71% (95% CI, 67-74%) and 92% (95% CI, 90-93%) performed by emergency physicians (EPs), 79% (95% CI, 71-85%) and 76% (95% CI, 69-81%) performed by surgeons, and 68% (95% CI 66-71%) and 87% (95% CI, 86-88%) performed by radiologists, respectively. There were no statistically significant differences among the three groups.ConclusionUS is a good imaging modality for the diagnosis of AC. EP-performed US has a similar diagnostic performance to radiologist-performed US. Further investigations would be needed to investigate the impact of US on expediting the management process and improving patient-centered outcomes.

Project description:ObjectiveTo assess and compare the performance of triage systems for identifying high and low-urgency patients in the emergency department (ED).DesignSystematic review and meta-analysis.Data sourcesEMBASE, Medline OvidSP, Cochrane central, Web of science and CINAHL databases from 1980 to 2016 with the final update in December 2018.Eligibility criteriaStudies that evaluated an emergency medical triage system, assessed validity using any reference standard as proxy for true patient urgency and were written in English. Studies conducted in low(er) income countries, based on case scenarios or involving less than 100 patients were excluded.Review methodsReviewers identified studies, extracted data and assessed the quality of the evidence independently and in duplicate. The Quality Assessment of studies of Diagnostic Accuracy included in Systematic Reviews -2 checklist was used to assess risk of bias. Raw data were extracted to create 2×2 tables and calculate sensitivity and specificity. ED patient volume and casemix severity of illness were investigated as determinants of triage systems' performance.ResultsSixty-six eligible studies evaluated 33 different triage systems. Comparisons were restricted to the three triage systems that had at least multiple evaluations using the same reference standard (Canadian Triage and Acuity Scale, Emergency Severity Index and Manchester Triage System). Overall, validity of each triage system to identify high and low-urgency patients was moderate to good, but performance was highly variable. In a subgroup analysis, no clear association was found between ED patient volume or casemix severity of illness and triage systems' performance.ConclusionsEstablished triage systems show a reasonable validity for the triage of patients at the ED, but performance varies considerably. Important research questions that remain are what determinants influence triage systems' performance and how the performance of existing triage systems can be improved.

Project description:ImportanceAn increasing number of machine learning (ML)-based clinical decision support systems (CDSSs) are described in the medical literature, but this research focuses almost entirely on comparing CDSS directly with clinicians (human vs computer). Little is known about the outcomes of these systems when used as adjuncts to human decision-making (human vs human with computer).ObjectivesTo conduct a systematic review to investigate the association between the interactive use of ML-based diagnostic CDSSs and clinician performance and to examine the extent of the CDSSs' human factors evaluation.Evidence reviewA search of MEDLINE, Embase, PsycINFO, and grey literature was conducted for the period between January 1, 2010, and May 31, 2019. Peer-reviewed studies published in English comparing human clinician performance with and without interactive use of an ML-based diagnostic CDSSs were included. All metrics used to assess human performance were considered as outcomes. The risk of bias was assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Risk of Bias in Non-Randomised Studies-Intervention (ROBINS-I). Narrative summaries were produced for the main outcomes. Given the heterogeneity of medical conditions, outcomes of interest, and evaluation metrics, no meta-analysis was performed.FindingsA total of 8112 studies were initially retrieved and 5154 abstracts were screened; of these, 37 studies met the inclusion criteria. The median number of participating clinicians was 4 (interquartile range, 3-8). Of the 107 results that reported statistical significance, 54 (50%) were increased by the use of CDSSs, 4 (4%) were decreased, and 49 (46%) showed no change or an unclear change. In the subgroup of studies carried out in representative clinical settings, no association between the use of ML-based diagnostic CDSSs and improved clinician performance could be observed. Interobserver agreement was the commonly reported outcome whose change was the most strongly associated with CDSS use. Four studies (11%) reported on user feedback, and, in all but 1 case, clinicians decided to override at least some of the algorithms' recommendations. Twenty-eight studies (76%) were rated as having a high risk of bias in at least 1 of the 4 QUADAS-2 core domains, and 6 studies (16%) were considered to be at serious or critical risk of bias using ROBINS-I.Conclusions and relevanceThis systematic review found only sparse evidence that the use of ML-based CDSSs is associated with improved clinician diagnostic performance. Most studies had a low number of participants, were at high or unclear risk of bias, and showed little or no consideration for human factors. Caution should be exercised when estimating the current potential of ML to improve human diagnostic performance, and more comprehensive evaluation should be conducted before deploying ML-based CDSSs in clinical settings. The results highlight the importance of considering supported human decisions as end points rather than merely the stand-alone CDSSs outputs.

Dataset Information

Diagnostic performance of electronic syndromic surveillance systems in acute care: a systematic review.

Context

Objective

Data sources

Study selection

Data extraction

Results

Conclusions

Publications

Diagnostic performance of electronic syndromic surveillance systems in acute care: a systematic review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets