Sepia, Tarsier, and Chameleon: A Modular C++ Framework for Event-Based Computer Vision.
Ontology highlight
ABSTRACT:
SUBMITTER: Marcireau A
PROVIDER: S-EPMC6960268 | biostudies-literature | 2019
REPOSITORIES: biostudies-literature
Publications
Sepia, Tarsier, and Chameleon: A Modular C++ Framework for Event-Based Computer Vision.
Marcireau Alexandre A Ieng Sio-Hoi SH Benosman Ryad R
Frontiers in neuroscience 20200108
This paper introduces an new open-source, header-only and modular C++ framework to facilitate the implementation of event-driven algorithms. The framework relies on three independent components: <i>sepia</i> (file IO), <i>tarsier</i> (algorithms), and <i>chameleon</i> (display). Our benchmarks show that algorithms implemented with <i>tarsier</i> are faster and have a lower latency than identical implementations in other state-of-the-art frameworks, thanks to static polymorphism (compile-time pip ...[more]
Similar Datasets
Project description:Motion segmentation is a critical pre-processing step for autonomous robotic systems to facilitate tracking of moving objects in cluttered environments. Event based sensors are low power analog devices that represent a scene by means of asynchronous information updates of only the dynamic details at high temporal resolution and, hence, require significantly less calculations. However, motion segmentation using spatiotemporal data is a challenging task due to data asynchrony. Prior approaches for object tracking using neuromorphic sensors perform well while the sensor is static or a known model of the object to be followed is available. To address these limitations, in this paper we develop a technique for generalized motion segmentation based on spatial statistics across time frames. First, we create micromotion on the platform to facilitate the separation of static and dynamic elements of a scene, inspired by human saccadic eye movements. Second, we introduce the concept of spike-groups as a methodology to partition spatio-temporal event groups, which facilitates computation of scene statistics and characterize objects in it. Experimental results show that our algorithm is able to classify dynamic objects with a moving camera with maximum accuracy of 92%.
| S-EPMC5334512 | biostudies-literature
Project description:Objective.Present methods for assessing color vision require the person's active participation. Here we describe a brain-computer interface-based method for assessing color vision that does not require the person's participation.Approach.This method uses steady-state visual evoked potentials to identify metamers-two light sources that have different spectral distributions but appear to the person to be the same color.Main results.We demonstrate that: minimization of the visual evoked potential elicited by two flickering light sources identifies the metamer; this approach can distinguish people with color-vision deficits from those with normal color vision; and this metamer-identification process can be automated.Significance.This new method has numerous potential clinical, scientific, and industrial applications.
| S-EPMC9094738 | biostudies-literature
Project description:Collisions arising from lane departures have contributed to traffic accidents causing millions of injuries and tens of thousands of casualties per year worldwide. Many related studies had shown that single vehicle lane departure crashes accounted largely in road traffic deaths that results from drifting out of the roadway. Hence, automotive safety has becoming a concern for the road users as most of the road casualties occurred due to driver's fallacious judgement of vehicle path. This paper proposes a vision-based lane departure warning framework for lane departure detection under daytime and night-time driving environments. The traffic flow and conditions of the road surface for both urban roads and highways in the city of Malacca are analysed in terms of lane detection rate and false positive rate. The proposed vision-based lane departure warning framework includes lane detection followed by a computation of a lateral offset ratio. The lane detection is composed of two stages: pre-processing and detection. In the pre-processing, a colour space conversion, region of interest extraction, and lane marking segmentation are carried out. In the subsequent detection stage, Hough transform is used to detect lanes. Lastly, the lateral offset ratio is computed to yield a lane departure warning based on the detected X-coordinates of the bottom end-points of each lane boundary in the image plane. For lane detection and lane departure detection performance evaluation, real-life datasets for both urban roads and highways in daytime and night-time driving environments, traffic flows, and road surface conditions are considered. The experimental results show that the proposed framework yields satisfactory results. On average, detection rates of 94.71% for lane detection rate and 81.18% for lane departure detection rate were achieved using the proposed frameworks. In addition, benchmark lane marking segmentation methods and Caltech lanes dataset were also considered for comparison evaluation in lane detection. Challenges to lane detection and lane departure detection such as worn lane markings, low illumination, arrow signs, and occluded lane markings are highlighted as the contributors to the false positive rates.
| S-EPMC6698973 | biostudies-literature
Project description:In order to safely navigate and orient in their local surroundings autonomous systems need to rapidly extract and persistently track visual features from the environment. While there are many algorithms tackling those tasks for traditional frame-based cameras, these have to deal with the fact that conventional cameras sample their environment with a fixed frequency. Most prominently, the same features have to be found in consecutive frames and corresponding features then need to be matched using elaborate techniques as any information between the two frames is lost. We introduce a novel method to detect and track line structures in data streams of event-based silicon retinae [also known as dynamic vision sensors (DVS)]. In contrast to conventional cameras, these biologically inspired sensors generate a quasicontinuous stream of vision information analogous to the information stream created by the ganglion cells in mammal retinae. All pixels of DVS operate asynchronously without a periodic sampling rate and emit a so-called DVS address event as soon as they perceive a luminance change exceeding an adjustable threshold. We use the high temporal resolution achieved by the DVS to track features continuously through time instead of only at fixed points in time. The focus of this work lies on tracking lines in a mostly static environment which is observed by a moving camera, a typical setting in mobile robotics. Since DVS events are mostly generated at object boundaries and edges which in man-made environments often form lines they were chosen as feature to track. Our method is based on detecting planes of DVS address events in x-y-t-space and tracing these planes through time. It is robust against noise and runs in real time on a standard computer, hence it is suitable for low latency robotics. The efficacy and performance are evaluated on real-world data sets which show artificial structures in an office-building using event data for tracking and frame data for ground-truth estimation from a DAVIS240C sensor.
| S-EPMC5825909 | biostudies-literature
Project description:PurposeTo evaluate the prevalence of computer vision syndrome (CVS)-related symptoms in a presbyopic population using the computer as the main work tool, as well as the relationship of CVS with the electronic device use habits and the ergonomic factors.MethodsA sample of 198 presbyopic participants (aged 45-65 years) who regularly work with a computer completed a customised questionnaire divided into: general demographics, optical correction commonly used and for work, habits of electronic devices use, ergonomic conditions during the working hours and CVS-related symptoms during work performance. A total of 10 CVS-related symptoms were questioned indicating the severity with which they occurred (0-4) and the median total symptom score (MTSS) was calculated as the sum of the symptoms.ResultsThe MTSS in this presbyopic population is 7 ± 5 symptoms. The most common symptoms reported by participants are dry eyes, tired eyes and difficulties in refocusing. MTSS is higher in women (p < 0.05), in laptop computer users (p < 0.05) and in teleworkers compared to office workers (p < 0.05). Regarding ergonomic conditions, MTSS is higher in participants who do not take breaks while working (p < 0.05), who have an inadequately lighting in the workspace (p < 0.05) and in the participants reporting neck (p < 0.01) or back pain (p < 0.001).ConclusionThere is a relationship between CVS-related symptoms, the use of electronic devices and the ergonomic factors, which indicates the importance of adapting workplaces, especially for home-based teleworkers, and following basic visual ergonomics rules.
| S-EPMC10133911 | biostudies-literature
Project description:Extracting meaning from a dynamic and variable flow of incoming information is a major goal of both natural and artificial intelligence. Computer vision (CV) guided by deep learning (DL) has made significant strides in recognizing a specific identity despite highly variable attributes. This is the same challenge faced by the nervous system and partially addressed by the concept cells-neurons exhibiting selective firing in response to specific persons/places, described in the human medial temporal lobe (MTL) . Yet, access to neurons representing a particular concept is limited due to these neurons' sparse coding. It is conceivable, however, that the information required for such decoding is present in relatively small neuronal populations. To evaluate how well neuronal populations encode identity information in natural settings, we recorded neuronal activity from multiple brain regions of nine neurosurgical epilepsy patients implanted with depth electrodes, while the subjects watched an episode of the TV series "24". First, we devised a minimally supervised CV algorithm (with comparable performance against manually-labeled data) to detect the most prevalent characters (above 1% overall appearance) in each frame. Next, we implemented DL models that used the time-varying population neural data as inputs and decoded the visual presence of the four main characters throughout the episode. This methodology allowed us to compare "computer vision" with "neuronal vision"-footprints associated with each character present in the activity of a subset of neurons-and identify the brain regions that contributed to this decoding process. We then tested the DL models during a recognition memory task following movie viewing where subjects were asked to recognize clip segments from the presented episode. DL model activations were not only modulated by the presence of the corresponding characters but also by participants' subjective memory of whether they had seen the clip segment, and by the associative strengths of the characters in the narrative plot. The described approach can offer novel ways to probe the representation of concepts in time-evolving dynamic behavioral tasks. Further, the results suggest that the information required to robustly decode concepts is present in the population activity of only tens of neurons even in brain regions beyond MTL.
| S-EPMC9837190 | biostudies-literature
Project description:Manual monitoring of animal behavior is time-consuming and prone to bias. An alternative to such limitations is using computational resources in behavioral assessments, such as tracking systems, to facilitate accurate and long-term evaluations. There is a demand for robust software that addresses analysis in heterogeneous environments (such as in field conditions) and evaluates multiple individuals in groups while maintaining their identities. The Ethoflow software was developed using computer vision and artificial intelligence (AI) tools to monitor various behavioral parameters automatically. An object detection algorithm based on instance segmentation was implemented, allowing behavior monitoring in the field under heterogeneous environments. Moreover, a convolutional neural network was implemented to assess complex behaviors expanding behavior analyses' possibilities. The heuristics used to generate training data for the AI models automatically are described, and the models trained with these datasets exhibited high accuracy in detecting individuals in heterogeneous environments and assessing complex behavior. Ethoflow was employed for kinematic assessments and to detect trophallaxis in social bees. The software was developed in desktop applications and had a graphical user interface. In the Ethoflow algorithm, the processing with AI is separate from the other modules, facilitating measurements on an ordinary computer and complex behavior assessing on machines with graphics processing units. Ethoflow is a useful support tool for applications in biology and related fields.
| S-EPMC8124799 | biostudies-literature
Project description:Tremor is one of the most common neurological symptoms. Its clinical and neurobiological complexity necessitates novel approaches for granular phenotyping. Instrumented neurophysiological analyses have proven useful, but are highly resource-intensive and lack broad accessibility. In contrast, bedside scores are simple to administer, but lack the granularity to capture subtle but relevant tremor features. We utilise the open-source computer vision pose tracking algorithm Mediapipe to track hands in clinical video recordings and use the resulting time series to compute canonical tremor features. This approach is compared to marker-based 3D motion capture, wrist-worn accelerometry, clinical scoring and a second, specifically trained tremor-specific algorithm in two independent clinical cohorts. These cohorts consisted of 66 patients diagnosed with essential tremor, assessed in different task conditions and states of deep brain stimulation therapy. We find that Mediapipe-derived tremor metrics exhibit high convergent clinical validity to scores (Spearman's ρ = 0.55-0.86, p≤ .01) as well as an accuracy of up to 2.60 mm (95% CI [-3.13, 8.23]) and ≤0.21 Hz (95% CI [-0.05, 0.46]) for tremor amplitude and frequency measurements, matching gold-standard equipment. Mediapipe, but not the disease-specific algorithm, was capable of analysing videos involving complex configurational changes of the hands. Moreover, it enabled the extraction of tremor features with diagnostic and prognostic relevance, a dimension which conventional tremor scores were unable to provide. Collectively, this demonstrates that current computer vision algorithms can be transformed into an accurate and highly accessible tool for video-based tremor analysis, yielding comparable results to gold standard tremor recordings.
| S-EPMC11192937 | biostudies-literature
Project description:State-of-the-art computer vision systems use frame-based cameras that sample the visual scene as a series of high-resolution images. These are then processed using convolutional neural networks using neurons with continuous outputs. Biological vision systems use a quite different approach, where the eyes (cameras) sample the visual scene continuously, often with a non-uniform resolution, and generate neural spike events in response to changes in the scene. The resulting spatio-temporal patterns of events are then processed through networks of spiking neurons. Such event-based processing offers advantages in terms of focusing constrained resources on the most salient features of the perceived scene, and those advantages should also accrue to engineered vision systems based upon similar principles. Event-based vision sensors, and event-based processing exemplified by the SpiNNaker (Spiking Neural Network Architecture) machine, can be used to model the biological vision pathway at various levels of detail. Here we use this approach to explore structural synaptic plasticity as a possible mechanism whereby biological vision systems may learn the statistics of their inputs without supervision, pointing the way to engineered vision systems with similar online learning capabilities.
| S-EPMC6015816 | biostudies-literature
Project description:Understanding the extremely variable, complex shape and venation characters of angiosperm leaves is one of the most challenging problems in botany. Machine learning offers opportunities to analyze large numbers of specimens, to discover novel leaf features of angiosperm clades that may have phylogenetic significance, and to use those characters to classify unknowns. Previous computer vision approaches have primarily focused on leaf identification at the species level. It remains an open question whether learning and classification are possible among major evolutionary groups such as families and orders, which usually contain hundreds to thousands of species each and exhibit many times the foliar variation of individual species. Here, we tested whether a computer vision algorithm could use a database of 7,597 leaf images from 2,001 genera to learn features of botanical families and orders, then classify novel images. The images are of cleared leaves, specimens that are chemically bleached, then stained to reveal venation. Machine learning was used to learn a codebook of visual elements representing leaf shape and venation patterns. The resulting automated system learned to classify images into families and orders with a success rate many times greater than chance. Of direct botanical interest, the responses of diagnostic features can be visualized on leaf images as heat maps, which are likely to prompt recognition and evolutionary interpretation of a wealth of novel morphological characters. With assistance from computer vision, leaves are poised to make numerous new contributions to systematic and paleobotanical studies.
| S-EPMC4812720 | biostudies-literature