Dataset Information

NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences.

ABSTRACT: Visual place recognition (VPR) is critical in not only localization and mapping for autonomous driving vehicles, but also assistive navigation for the visually impaired population. To enable a long-term VPR system on a large scale, several challenges need to be addressed. First, different applications could require different image view directions, such as front views for self-driving cars while side views for the low vision people. Second, VPR in metropolitan scenes can often cause privacy concerns due to the imaging of pedestrian and vehicle identity information, calling for the need for data anonymization before VPR queries and database construction. Both factors could lead to VPR performance variations that are not well understood yet. To study their influences, we present the NYU-VPR dataset that contains more than 200,000 images over a 2km×2km area near the New York University campus, taken within the whole year of 2016. We present benchmark results on several popular VPR algorithms showing that side views are significantly more challenging for current VPR methods while the influence of data anonymization is almost negligible, together with our hypothetical explanations and in-depth analysis.

SUBMITTER: Sheng D

PROVIDER: S-EPMC9394449 | biostudies-literature | 2021 Sep-Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences.

Sheng Diwei D Chai Yuxiang Y Li Xinru X Feng Chen C Lin Jianzhe J Silva Claudio C Rizzo John-Ross JR

Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE/RSJ International Conference on Intelligent Robots and Systems 20210901

Visual place recognition (VPR) is critical in not only localization and mapping for autonomous driving vehicles, but also assistive navigation for the visually impaired population. To enable a long-term VPR system on a large scale, several challenges need to be addressed. First, different applications could require different image view directions, such as front views for self-driving cars while side views for the low vision people. Second, VPR in metropolitan scenes can often cause privacy conce ...[more]

PMID: 35999910

Similar Datasets

Project description:Visual place recognition (VPR) is the ability to recognize locations in a physical environment based only on visual inputs. It is a challenging task due to perceptual aliasing, viewpoint and appearance variations and complexity of dynamic scenes. Despite promising demonstrations, many state-of-the-art (SOTA) VPR approaches based on artificial neural networks (ANNs) suffer from computational inefficiency. However, spiking neural networks (SNNs) implemented on neuromorphic hardware are reported to have remarkable potential for more efficient solutions computationally. Still, training SOTA SNNs for VPR is often intractable on large and diverse datasets, and they typically demonstrate poor real-time operation performance. To address these shortcomings, we developed an end-to-end convolutional SNN model for VPR that leverages backpropagation for tractable training. Rate-based approximations of leaky integrate-and-fire (LIF) neurons are employed during training, which are then replaced with spiking LIF neurons during inference. The proposed method significantly outperforms existing SOTA SNNs on challenging datasets like Nordland and Oxford RobotCar, achieving 78.6% precision at 100% recall on the Nordland dataset (compared to 73.0% from the current SOTA) and 45.7% on the Oxford RobotCar dataset (compared to 20.2% from the current SOTA). Our approach offers a simpler training pipeline while yielding significant improvements in both training and inference times compared to SOTA SNNs for VPR. Hardware-in-the-loop tests using Intel's neuromorphic USB form factor, Kapoho Bay, show that our on-chip spiking models for VPR trained via the ANN-to-SNN conversion strategy continue to outperform their SNN counterparts, despite a slight but noticeable decrease in performance when transitioning from off-chip to on-chip, while offering significant energy efficiency. The results highlight the outstanding rapid prototyping and real-world deployment capabilities of this approach, showing it to be a substantial step toward more prevalent SNN-based real-world robotics solutions.

Project description:During manipulation, object slipping is prevented by modulating the grip force (GF) in synchrony with motion-related inertial forces, i.e., load force (LF). However, due to conduction delays of the sensory system, GF must be modulated in advance based on predictions of LF changes. It has been proposed that such predictive force control relies on internal representations, i.e., internal models, of the relation between the dynamic of the environment and movement kinematics. Somatosensory and visual feedback plays a primary role in building these internal representations. For instance, it has been shown that manipulation-dependent somatosensory signals contribute to building internal representations of gravity in normal and altered gravitational contexts. Furthermore, delaying the timing of visual feedback of object displacement has been shown to affect GF. Here, we explored whether and the extent to which spatial features of visual feedback movement, such as motion direction, may contribute to GF control. If this were the case, a spatial mismatch between actual (somatosensory) and visual feedback of object motion would elicit changes in GF modulation. We tested this hypothesis by asking participants to generate vertical object movements while visual feedback of object position was congruent (0° rotation) or incongruent (180° or 90°) with the actual object displacement. The role of vision on GF control was quantified by the temporal shift of GF modulation as a function of visual feedback orientation and actual object motion direction. GF control was affected by visual feedback when this was incongruent in the vertical (180°), but not horizontal dimension. Importantly, 180° visual feedback rotation delayed and anticipated GF modulation during upward and downward actual movements, respectively. Our findings suggest that during manipulation, spatial features of visual feedback motion are used to predict upcoming LF changes. Furthermore, the present study provides evidence that an internal model of gravity contributes to GF control by influencing sensory reweighting processes during object manipulation.

Dataset Information

NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences.

Publications

NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets