Event-Based Face Detection and Tracking Using the Dynamics of Eye Blinks.
Ontology highlight
ABSTRACT:
SUBMITTER: Lenz G
PROVIDER: S-EPMC7397845 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
Event-Based Face Detection and Tracking Using the Dynamics of Eye Blinks.
Lenz Gregor G Ieng Sio-Hoi SH Benosman Ryad R
Frontiers in neuroscience 20200727
We present the first purely event-based method for face detection using the high temporal resolution properties of an event-based camera to detect the presence of a face in a scene using eye blinks. Eye blinks are a unique and stable natural dynamic temporal signature of human faces across population that can be fully captured by event-based sensors. We show that eye blinks have a unique temporal signature over time that can be easily detected by correlating the acquired local activity with a ge ...[more]
Similar Datasets
Project description:In face-to-face communication, recurring intervals of mutual gaze allow listeners to provide speakers with visual feedback (e.g. nodding). Here, we investigate the potential feedback function of one of the subtlest of human movements-eye blinking. While blinking tends to be subliminal, the significance of mutual gaze in human interaction raises the question whether the interruption of mutual gaze through blinking may also be communicative. To answer this question, we developed a novel, virtual reality-based experimental paradigm, which enabled us to selectively manipulate blinking in a virtual listener, creating small differences in blink duration resulting in 'short' (208 ms) and 'long' (607 ms) blinks. We found that speakers unconsciously took into account the subtle differences in listeners' blink duration, producing substantially shorter answers in response to long listener blinks. Our findings suggest that, in addition to physiological, perceptual and cognitive functions, listener blinks are also perceived as communicative signals, directly influencing speakers' communicative behavior in face-to-face communication. More generally, these findings may be interpreted as shedding new light on the evolutionary origins of mental-state signaling, which is a crucial ingredient for achieving mutual understanding in everyday social interaction.
| S-EPMC6291193 | biostudies-literature
Project description:To detect and quantify eye blinks during 24-hour intraocular pressure (IOP) monitoring with a contact lens sensor (CLS).A total of 249 recordings of 24-hour IOP patterns from 202 participants using a CLS were included. Software was developed to automatically detect eye blinks, and wake and sleep periods. The blink detection method was based on detection of CLS signal peaks greater than a threshold proportional to the signal amplitude. Three methods for automated detection of the sleep and wake periods were evaluated. These relied on blink detection and subsequent comparison of the local signal amplitude with a threshold proportional to the mean signal amplitude. These methods were compared to manual sleep/wake verification. In a pilot, simultaneous video recording of 10 subjects was performed to compare the software to observer-measured blink rates.Mean (SD) age of participants was 57.4 ± 16.5 years (males, 49.5%). There was excellent agreement between software-detected number of blinks and visually measured blinks for both observers (intraclass correlation coefficient [ICC], 0.97 for observer 1; ICC, 0.98 for observer 2). The CLS measured a mean blink frequency of 29.8 ± 15.4 blinks/min, a blink duration of 0.26 ± 0.21 seconds and an interblink interval of 1.91 ± 2.03 seconds. The best method for identifying sleep periods had an accuracy of 95.2 ± 0.5%.Automated analysis of CLS 24-hour IOP recordings can accurately quantify eye blinks, and identify sleep and wake periods.This study sheds new light on the potential importance of eye blinks in glaucoma and may contribute to improved understanding of circadian IOP characteristics.
| S-EPMC4306263 | biostudies-other
Project description:We blink more often than required for maintaining the corneal tear film. Whether there are perceptual or cognitive consequences of blinks that may justify their high frequency is unclear. Previous findings showed that blinks may indicate switches between large-scale cortical networks, such as dorsal attention and default-mode networks. Thus, blinks may trigger a refresh of visual attention. Yet, this has so far not been confirmed behaviorally. Here, we tested the effect of blinks on visual performance in a series of rapid serial visual presentation tasks. In Experiment 1, participants had to identify a target digit embedded in a random stream of letter distractors, presented foveally for 60 ms each. Participants blinked once during the presentation stream. In a separate condition, blinks were simulated by shutter glasses. Detection performance was enhanced (up to 13% point increase in accuracy) for targets appearing up to 300 ms after eye blinks. Performance boosts were stronger for voluntary blinks than artificial blinks. This performance boost was also replicated with more naturalistic stimuli (Experiment 2). We conclude that eye blinks lead to attentional benefits for object recognition in the period after reopening of the eyelids and may be used strategically for temporarily boosting visual performance.
| S-EPMC7545084 | biostudies-literature
Project description:BackgroundCriminal associates such as terrorist members are likely to deny knowing members of their network when questioned by police. Eye tracking research suggests that lies about familiar faces can be detected by distinct markers of recognition (e.g. fewer fixations and longer fixation durations) across multiple eye fixation parameters. However, the effect of explicit eye movement strategies to concealed recognition on such markers has not been examined. Our aim was to assess the impact of fixed-sequence eye movement strategies (across the forehead, ears, eyes, nose, mouth and chin) on markers of familiar face recognition. Participants were assigned to one of two groups: a standard guilty group who were simply instructed to conceal knowledge but with no specific instructions on how to do so; and a countermeasures group who were instructed to look at every familiar and unfamiliar face in the same way by executing a consistent sequence of fixations.ResultsIn the standard guilty group, lies about recognition of familiar faces showed longer average fixation durations, a lower proportion of fixations to the inner face regions, and proportionately more viewing of the eyes than honest responses to genuinely unknown faces. In the countermeasures condition, familiar face recognition was detected by longer fixations durations, fewer fixations to the inner regions of the face, and fewer interest areas of the face viewed. Longer fixation durations were a consistent marker of recognition across both conditions for most participants; differences were detectable from the first fixation.ConclusionThe results suggest that individuals can exert a degree of executive control over fixation patterns but that: the eyes are particularly attention-grabbing for familiar faces; the more viewers look around the face, the more they give themselves away; and attempts to deploy the same fixation patterns to familiar and unfamiliar faces were unsuccessful. The results suggest that the best strategy for concealing recognition might be to keep the eyes fixated in the centre of the screen but, even then, recognition is apparent in longer fixation durations. We discuss potential optimal conditions for detecting concealed knowledge of faces.
| S-EPMC6684707 | biostudies-literature
Project description:Self-face recognition has been shown to be impaired in schizophrenia (SZ), according to studies using behavioral tasks implicating cognitive demands. Here, we employed an eye-tracking methodology, which is a relevant tool to understand impairments in self-face recognition deficits in SZ because it provides a natural, continuous and online record of face processing. Moreover, it allows collecting the most relevant and informative features each individual looks at during the self-face recognition. These advantages are especially relevant considering the fundamental role played by the patterns of visual exploration on face processing. Thus, this paper aims to investigate self-face recognition deficits in SZ using eye-tracking methodology. Visual scan paths were monitored in 20 patients with SZ and 20 healthy controls. Self, famous, and unknown faces were morphed in steps of 20%. Location, number, and duration of fixations on relevant areas were recorded with an eye-tracking system. Participants performed a passive exploration task (no specific instruction was provided), followed by an active decision making task (individuals were explicitly requested to recognize the different faces). Results showed that patients with SZ had fewer and longer fixations compared to controls. Nevertheless, both groups focused their attention on relevant facial features in a similar way. No significant difference was found between groups when participants were requested to recognize the faces (active task). In conclusion, using an eye tracking methodology and two tasks with low levels of cognitive demands, our results suggest that patients with SZ are able to: (1) explore faces and focus on relevant features of the face in a similar way as controls; and (2) recognize their own face.
| S-EPMC4748053 | biostudies-literature
Project description:ObjectiveTo develop new standardized eye tracking based measures and metrics for infants' gaze dynamics in the face-distractor competition paradigm.MethodEye tracking data were collected from two samples of healthy 7-month-old (total n?=?45), as well as one sample of 5-month-old infants (n?=?22) in a paradigm with a picture of a face or a non-face pattern as a central stimulus, and a geometric shape as a lateral stimulus. The data were analyzed by using conventional measures of infants' initial disengagement from the central to the lateral stimulus (i.e., saccadic reaction time and probability) and, additionally, novel measures reflecting infants gaze dynamics after the initial disengagement (i.e., cumulative allocation of attention to the central vs. peripheral stimulus).ResultsThe results showed that the initial saccade away from the centrally presented stimulus is followed by a rapid re-engagement of attention with the central stimulus, leading to cumulative preference for the central stimulus over the lateral stimulus over time. This pattern tended to be stronger for salient facial expressions as compared to non-face patterns, was replicable across two independent samples of 7-month-old infants, and differentiated between 7 and 5 month-old infants.ConclusionThe results suggest that eye tracking based assessments of infants' cumulative preference for faces over time can be readily parameterized and standardized, and may provide valuable techniques for future studies examining normative developmental changes in preference for social signals.SignificanceStandardized measures of early developing face preferences may have potential to become surrogate biomarkers of neurocognitive and social development.
| S-EPMC4028213 | biostudies-literature
Project description:Face gaze is a fundamental non-verbal behaviour and can be assessed using eye-tracking glasses. Methodological guidelines are lacking on which measure to use to determine face gaze. To evaluate face gaze patterns we compared three measures: duration, frequency and dwell time. Furthermore, state of the art face gaze analysis requires time and manual effort. We tested if face gaze patterns in the first 30, 60 and 120 s predict face gaze patterns in the remaining interaction. We performed secondary analyses of mobile eye-tracking data of 16 internal medicine physicians in consultation with 100 of their patients. Duration and frequency of face gaze were unrelated. The lack of association between duration and frequency suggests that research may yield different results depending on which measure of face gaze is used. Dwell time correlates both duration and frequency. Face gaze during the first seconds of the consultations predicted face gaze patterns of the remaining consultation time (R2 0.26 to 0.73). Therefore, face gaze during the first minutes of the consultations can be used to predict face gaze patterns over the complete interaction. Researchers interested to study face gaze may use these findings to make optimal methodological choices.
| S-EPMC8494763 | biostudies-literature
Project description:In order to safely navigate and orient in their local surroundings autonomous systems need to rapidly extract and persistently track visual features from the environment. While there are many algorithms tackling those tasks for traditional frame-based cameras, these have to deal with the fact that conventional cameras sample their environment with a fixed frequency. Most prominently, the same features have to be found in consecutive frames and corresponding features then need to be matched using elaborate techniques as any information between the two frames is lost. We introduce a novel method to detect and track line structures in data streams of event-based silicon retinae [also known as dynamic vision sensors (DVS)]. In contrast to conventional cameras, these biologically inspired sensors generate a quasicontinuous stream of vision information analogous to the information stream created by the ganglion cells in mammal retinae. All pixels of DVS operate asynchronously without a periodic sampling rate and emit a so-called DVS address event as soon as they perceive a luminance change exceeding an adjustable threshold. We use the high temporal resolution achieved by the DVS to track features continuously through time instead of only at fixed points in time. The focus of this work lies on tracking lines in a mostly static environment which is observed by a moving camera, a typical setting in mobile robotics. Since DVS events are mostly generated at object boundaries and edges which in man-made environments often form lines they were chosen as feature to track. Our method is based on detecting planes of DVS address events in x-y-t-space and tracing these planes through time. It is robust against noise and runs in real time on a standard computer, hence it is suitable for low latency robotics. The efficacy and performance are evaluated on real-world data sets which show artificial structures in an office-building using event data for tracking and frame data for ground-truth estimation from a DAVIS240C sensor.
| S-EPMC5825909 | biostudies-literature
Project description:Many eye tracking studies use facial stimuli presented on a display to investigate attentional processing of social stimuli. To introduce a more realistic approach that allows interaction between two real people, we evaluated a new eye tracking setup in three independent studies in terms of data quality, short-term reliability and feasibility. Study 1 measured the robustness, precision and accuracy for calibration stimuli compared to a classical display-based setup. Study 2 used the identical measures with an independent study sample to compare the data quality for a photograph of a face (2D) and the face of the real person (3D). Study 3 evaluated data quality over the course of a real face-to-face conversation and examined the gaze behavior on the facial features of the conversation partner. Study 1 provides evidence that quality indices for the scene-based setup were comparable to those of a classical display-based setup. Average accuracy was better than 0.4° visual angle. Study 2 demonstrates that eye tracking quality is sufficient for 3D stimuli and robust against short interruptions without re-calibration. Study 3 confirms the long-term stability of tracking accuracy during a face-to-face interaction and demonstrates typical gaze patterns for facial features. Thus, the eye tracking setup presented here seems feasible for studying gaze behavior in dyadic face-to-face interactions. Eye tracking data obtained with this setup achieves an accuracy that is sufficient for investigating behavior such as eye contact in social interactions in a range of populations including clinical conditions, such as autism spectrum and social phobia.
| S-EPMC7846602 | biostudies-literature
Project description:Effective working memory (WM) functioning depends on the gating process that regulates the balance between maintenance and updating of WM. The present study used the event-based eye-blink rate (ebEBR), which presumably reflects phasic striatal dopamine activity, to examine how the cognitive processes of gating and updating separately facilitate flexible updating of WM contents and the potential involvement of dopamine in these processes. Real-time changes in eye blinks were tracked during performance on the reference-back task, in which demands on these two processes were independently manipulated. In all three experiments, trials that required WM updating and trials that required gate switching were both associated with increased ebEBR. These results may support the prefrontal cortex basal ganglia WM model (PBWM) by linking updating and gating to striatal dopaminergic activity. In Experiment 3, the ebEBR was used to determine what triggers gate switching. We found that switching to an updating mode (gate opening) was more stimulus driven and retroactive than switching to a maintenance mode, which was more context driven. Together, these findings show that the ebEBR - an inexpensive, non-invasive, easy-to-use measure - can be used to track changes in WM demands during task performance and, hence, possibly striatal dopamine activity.
| S-EPMC5451427 | biostudies-other