Dataset Information

Detecting Arbitrarily Oriented Subspace Clusters in Data Streams Using Hough Transform

ABSTRACT: When facing high-dimensional data streams, clustering algorithms quickly reach the boundaries of their usefulness as most of these methods are not designed to deal with the curse of dimensionality. Due to inherent sparsity in high-dimensional data, distances between objects tend to become meaningless since the distances between any two objects measured in the full dimensional space tend to become the same for all pairs of objects. In this work, we present a novel oriented subspace clustering algorithm that is able to deal with such issues and detects arbitrarily oriented subspace clusters in high-dimensional data streams. Data streams generally implicate the challenge that the data cannot be stored entirely and hence there is a general demand for suitable data handling strategies for clustering algorithms such that the data can be processed within a single scan. We therefore propose the CashStream algorithm that unites state-of-the-art stream processing techniques and additionally relies on the Hough transform to detect arbitrarily oriented subspace clusters. Our experiments compare CashStream to its static counterpart and show that the amount of consumed memory is significantly decreased while there is no loss in terms of runtime. Electronic supplementary material The online version of this chapter (10.1007/978-3-030-47426-3_28) contains supplementary material, which is available to authorized users.

SUBMITTER: Lauw H

PROVIDER: S-EPMC7206268 | biostudies-literature | 2020 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundPassive reporting and laboratory testing delays may limit gastrointestinal (GI) disease outbreak detection. Healthcare systems routinely collect clinical data in electronic medical records (EMRs) that could be used for surveillance. This study's primary objective was to identify data streams from EMRs that may perform well for GI outbreak detection.MethodsZip code-specific daily episode counts in 2009 were generated for 22 syndromic and laboratory-based data streams from Kaiser Permanente Northern California EMRs, covering 3.3 million members. Data streams included outpatient and inpatient diagnosis codes, antidiarrheal medication dispensings, stool culture orders, and positive microbiology tests for six GI pathogens. Prospective daily surveillance was mimicked using the space-time permutation scan statistic in single and multi-stream analyses, and space-time clusters were identified. Serotype relatedness was assessed for isolates in two Salmonella clusters.ResultsPotential outbreaks included a cluster of 18 stool cultures ordered over 5 days in one zip code and a Salmonella cluster in three zip codes over 9 days, in which at least five of six cases had the same rare serotype. In all, 28 potential outbreaks were identified using single stream analyses, with signals in outpatient diagnosis codes most common. Multi-stream analyses identified additional potential outbreaks and in one example, improved the timeliness of detection.ConclusionsGI disease-related data streams can be used to identify potential outbreaks when generated from EMRs with extensive regional coverage. This process can supplement traditional GI outbreak reports to health departments, which frequently consist of outbreaks in well-defined settings (e.g., day care centers and restaurants) with no laboratory-confirmed pathogen. Data streams most promising for surveillance included microbiology test results, stool culture orders, and outpatient diagnoses. In particular, clusters of microbiology tests positive for specific pathogens could be identified in EMRs and used to prioritize further testing at state health departments, potentially improving outbreak detection.

Dataset Information

Detecting Arbitrarily Oriented Subspace Clusters in Data Streams Using Hough Transform

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets