Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Back to the feature : Learning robust camera localization from pixels to pose

Sarlin, Paul-Edouard ; Unagar, Ajaykumar ; Larsson, Mans ; Germain, Hugo ; Toft, Carl ; Larsson, Viktor LU ; Pollefeys, Marc ; Lepetit, Vincent ; Hammarstrand, Lars and Kahl, Fredrik LU (2021) 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021 p.3246-3256
Abstract
Camera pose estimation in known scenes is a 3D geometry task recently tackled by multiple learning algorithms. Many regress precise geometric quantities, like poses or 3D points, from an input image. This either fails to generalize to new viewpoints or ties the model parameters to a specific scene. In this paper, we go Back to the Feature: we argue that deep networks should focus on learning robust and invariant visual features, while the geometric estimation should be left to principled algorithms. We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model. Our approach is based on the direct alignment of multiscale deep features, casting camera localization as metric learning.... (More)
Camera pose estimation in known scenes is a 3D geometry task recently tackled by multiple learning algorithms. Many regress precise geometric quantities, like poses or 3D points, from an input image. This either fails to generalize to new viewpoints or ties the model parameters to a specific scene. In this paper, we go Back to the Feature: we argue that deep networks should focus on learning robust and invariant visual features, while the geometric estimation should be left to principled algorithms. We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model. Our approach is based on the direct alignment of multiscale deep features, casting camera localization as metric learning. PixLoc learns strong data priors by end-to-end training from pixels to pose and exhibits exceptional generalization to new scenes by separating model parameters and scene geometry. The system can localize in large environments given coarse pose priors but also improve the accuracy of sparse feature matching by jointly refining keypoints and poses with little overhead. The code will be publicly available at github.com/cvg/pixloc. (Less)
Please use this url to cite or link to this publication:
author
; ; ; ; ; ; ; ; and
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
pages
11 pages
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
conference name
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2021
conference location
Virtual, Online, United States
conference dates
2021-06-19 - 2021-06-25
external identifiers
  • scopus:85119000929
DOI
10.1109/CVPR46437.2021.00326
language
English
LU publication?
no
id
82c957bb-d544-401b-83a0-665884c5b903
date added to LUP
2022-09-06 13:18:25
date last changed
2022-09-20 17:59:04
@inproceedings{82c957bb-d544-401b-83a0-665884c5b903,
  abstract     = {{Camera pose estimation in known scenes is a 3D geometry task recently tackled by multiple learning algorithms. Many regress precise geometric quantities, like poses or 3D points, from an input image. This either fails to generalize to new viewpoints or ties the model parameters to a specific scene. In this paper, we go Back to the Feature: we argue that deep networks should focus on learning robust and invariant visual features, while the geometric estimation should be left to principled algorithms. We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model. Our approach is based on the direct alignment of multiscale deep features, casting camera localization as metric learning. PixLoc learns strong data priors by end-to-end training from pixels to pose and exhibits exceptional generalization to new scenes by separating model parameters and scene geometry. The system can localize in large environments given coarse pose priors but also improve the accuracy of sparse feature matching by jointly refining keypoints and poses with little overhead. The code will be publicly available at github.com/cvg/pixloc.}},
  author       = {{Sarlin, Paul-Edouard and Unagar, Ajaykumar and Larsson, Mans and Germain, Hugo and Toft, Carl and Larsson, Viktor and Pollefeys, Marc and Lepetit, Vincent and Hammarstrand, Lars and Kahl, Fredrik}},
  booktitle    = {{2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}},
  language     = {{eng}},
  pages        = {{3246--3256}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  title        = {{Back to the feature : Learning robust camera localization from pixels to pose}},
  url          = {{http://dx.doi.org/10.1109/CVPR46437.2021.00326}},
  doi          = {{10.1109/CVPR46437.2021.00326}},
  year         = {{2021}},
}