Published on Thu May 10 2018

Structure-from-Motion using Dense CNN Features with Keypoint Relocalization

Aji Resindra Widya, Akihiko Torii, Masatoshi Okutomi

Structure from Motion (SfM) using imagery that involves extreme appearance changes is yet a challenging task due to a loss of feature repeatability. Using densely extracted convolutional neural network (CNN) features significantly improves the SfM reconstruction capability.

0
0
0
Abstract

Structure from Motion (SfM) using imagery that involves extreme appearance changes is yet a challenging task due to a loss of feature repeatability. Using feature correspondences obtained by matching densely extracted convolutional neural network (CNN) features significantly improves the SfM reconstruction capability. However, the reconstruction accuracy is limited by the spatial resolution of the extracted CNN features which is not even pixel-level accuracy in the existing approach. Providing dense feature matches with precise keypoint positions is not trivial because of memory limitation and computational burden of dense features. To achieve accurate SfM reconstruction with highly repeatable dense features, we propose an SfM pipeline that uses dense CNN features with relocalization of keypoint position that can efficiently and accurately provide pixel-level feature correspondences. Then, we demonstrate on the Aachen Day-Night dataset that the proposed SfM using dense CNN features with the keypoint relocalization outperforms a state-of-the-art SfM (COLMAP using RootSIFT) by a large margin.

Wed Aug 18 2021our pick
Computer Vision
Pixel-Perfect Structure-from-Motion with Featuremetric Refinement
Finding local features that are repeatable across multiple views is acornerstone of sparse 3D reconstruction. The classical image matching paradigm detects keypoints per-image once and for all. This can yield poorly-localized features and propagate large errors to the final geometry.
4
36
202
Tue Jul 17 2018
Computer Vision
GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints
Learned local descriptors based on Convolutional Neural Networks (CNNs) have achieved significant improvements on patch-based benchmarks. Not having strong generalization ability on recent benchmarks of image-based reconstruction is a limitation. In this paper, we propose avel local descriptor learning approach that integrates geometry
0
0
0
Mon Mar 30 2020
Computer Vision
DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learning
0
0
0
Thu May 24 2018
Computer Vision
LF-Net: Learning Local Features from Images
We present a novel deep architecture and a training strategy to learn a local feature pipeline from scratch. We exploit depth and relative camera pose cues to create a virtual target that the network should achieve on one image. While this process isherently non-differentiable, we show that we can optimize
0
0
0
Sat Aug 03 2019
Computer Vision
Learning Local Feature Descriptor with Motion Attribute for Vision-based Localization
The motion attribute of a local feature point could be exploited to improve localization performance. The proposed algorithm can be integrated into a vision-based localization algorithm to improve estimation accuracy significantly.
0
0
0
Thu May 09 2019
Computer Vision
D2-Net: A Trainable CNN for Joint Detection and Description of Local Features
A single convolutional neural network plays a dual role: It is simultaneously a dense feature descriptor and a feature detector. The proposed method obtains state-of-the-art performance on both the difficult Aachen Day-Night localization dataset and the InLoc indoor localization benchmark.
0
0
0