Dataset Information

MEMIS: Multimodal Emergency Management Information System

ABSTRACT: The recent upsurge in the usage of social media and the multimedia data generated therein has attracted many researchers for analyzing and decoding the information to automate decision-making in several fields. This work focuses on one such application: disaster management in times of crises and calamities. The existing research on disaster damage analysis has primarily taken only unimodal information in the form of text or image into account. These unimodal systems, although useful, fail to model the relationship between the various modalities. Different modalities often present supporting facts about the task, and therefore, learning them together can enhance performance. We present MEMIS, a system that can be used in emergencies like disasters to identify and analyze the damage indicated by user-generated multimodal social media posts, thereby helping the disaster management groups in making informed decisions. Our leave-one-disaster-out experiments on a multimodal dataset suggest that not only does fusing information in different media forms improves performance, but that our system can also generalize well to new disaster categories. Further qualitative analysis reveals that the system is responsive and computationally efficient.

SUBMITTER: Jose J

PROVIDER: S-EPMC7148216 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:A ubiquitous feature of the nervous system is the processing of simultaneously arriving sensory inputs from different modalities. Yet, because of the difficulties of monitoring large populations of neurons with the single resolution required to determine their sensory responses, the cellular mechanisms of how populations of neurons encode different sensory modalities often remain enigmatic. We studied multimodal information encoding in a small sensorimotor system of the crustacean stomatogastric nervous system that drives rhythmic motor activity for the processing of food. This system is experimentally advantageous, as it produces a fictive behavioral output in vitro, and distinct sensory modalities can be selectively activated. It has the additional advantage that all sensory information is routed through a hub ganglion, the commissural ganglion, a structure with fewer than 220 neurons. Using optical imaging of a population of commissural neurons to track each individual neuron's response across sensory modalities, we provide evidence that multimodal information is encoded via a combinatorial code of recruited neurons. By selectively stimulating chemosensory and mechanosensory inputs that are functionally important for processing of food, we find that these two modalities were processed in a distributed network comprising the majority of commissural neurons imaged. In a total of 12 commissural ganglia, we show that 98% of all imaged neurons were involved in sensory processing, with the two modalities being processed by a highly overlapping set of neurons. Of these, 80% were multimodal, 18% were unimodal, and only 2% of the neurons did not respond to either modality. Differences between modalities were represented by the identities of the neurons participating in each sensory condition and by differences in response sign (excitation versus inhibition), with 46% changing their responses in the other modality. Consistent with the hypothesis that the commissural network encodes different sensory conditions in the combination of activated neurons, a new combination of excitation and inhibition was found when both pathways were activated simultaneously. The responses to this bimodal condition were distinct from either unimodal condition, and for 30% of the neurons, they were not predictive from the individual unimodal responses. Thus, in a sensorimotor network, different sensory modalities are encoded using a combinatorial code of neurons that are activated or inhibited. This provides motor networks with the ability to differentially respond to categorically different sensory conditions and may serve as a model to understand higher-level processing of multimodal information.

Dataset Information

MEMIS: Multimodal Emergency Management Information System

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets