Robust Incremental Structure-from-Motion with Hybrid Features
(2025) 18th European Conference on Computer Vision, ECCV 2024 In Lecture Notes in Computer Science 15094 LNCS. p.249-269- Abstract
Structure-from-Motion (SfM) has become a ubiquitous tool for camera calibration and scene reconstruction with many downstream applications in computer vision and beyond. While the state-of-the-art SfM pipelines have reached a high level of maturity in well-textured and well-configured scenes over the last decades, they still fall short of robustly solving the SfM problem in challenging scenarios. In particular, weakly textured scenes and poorly constrained configurations oftentimes cause catastrophic failures or large errors for the primarily keypointbased pipelines. In these scenarios, line segments are often abundant and can offer complementary geometric constraints. Their large spatial extent and typically structured configurations... (More)
Structure-from-Motion (SfM) has become a ubiquitous tool for camera calibration and scene reconstruction with many downstream applications in computer vision and beyond. While the state-of-the-art SfM pipelines have reached a high level of maturity in well-textured and well-configured scenes over the last decades, they still fall short of robustly solving the SfM problem in challenging scenarios. In particular, weakly textured scenes and poorly constrained configurations oftentimes cause catastrophic failures or large errors for the primarily keypointbased pipelines. In these scenarios, line segments are often abundant and can offer complementary geometric constraints. Their large spatial extent and typically structured configurations lead to stronger geometric constraints as compared to traditional keypoint-based methods. In this work, we introduce an incremental SfM system that, in addition to points, leverages lines and their structured geometric relations. Our technical contributions span the entire pipeline (mapping, triangulation, registration) and we integrate these into a comprehensive end-to-end SfM system that we share as an open-source software with the community. We also present the first analytical method to propagate uncertainties for 3D optimized lines via sensitivity analysis. Experiments show that our system is consistently more robust and accurate compared to the widely used point-based state of the art in SfM – achieving richer maps and more precise camera registrations, especially under challenging conditions. In addition, our uncertainty-aware localization module alone is able to consistently improve over the state of the art under both pointalone and hybrid setups.
(Less)
- author
- Liu, Shaohui ; Gao, Yidan ; Zhang, Tianyi ; Pautrat, Rémi ; Schönberger, Johannes L. ; Larsson, Viktor LU and Pollefeys, Marc
- organization
- publishing date
- 2025
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- host publication
- Computer Vision – ECCV 2024 - 18th European Conference, Proceedings
- series title
- Lecture Notes in Computer Science
- editor
- Leonardis, Aleš ; Ricci, Elisa ; Roth, Stefan ; Russakovsky, Olga ; Sattler, Torsten and Varol, Gül
- volume
- 15094 LNCS
- pages
- 21 pages
- publisher
- Springer Science and Business Media B.V.
- conference name
- 18th European Conference on Computer Vision, ECCV 2024
- conference location
- Milan, Italy
- conference dates
- 2024-09-29 - 2024-10-04
- external identifiers
-
- scopus:105018217825
- ISSN
- 1611-3349
- 0302-9743
- ISBN
- 9783031727634
- DOI
- 10.1007/978-3-031-72764-1_15
- language
- English
- LU publication?
- yes
- id
- fdf80e96-c92b-48cf-b444-de28ec4802bc
- date added to LUP
- 2025-11-28 11:26:12
- date last changed
- 2025-11-28 11:27:29
@inproceedings{fdf80e96-c92b-48cf-b444-de28ec4802bc,
abstract = {{<p>Structure-from-Motion (SfM) has become a ubiquitous tool for camera calibration and scene reconstruction with many downstream applications in computer vision and beyond. While the state-of-the-art SfM pipelines have reached a high level of maturity in well-textured and well-configured scenes over the last decades, they still fall short of robustly solving the SfM problem in challenging scenarios. In particular, weakly textured scenes and poorly constrained configurations oftentimes cause catastrophic failures or large errors for the primarily keypointbased pipelines. In these scenarios, line segments are often abundant and can offer complementary geometric constraints. Their large spatial extent and typically structured configurations lead to stronger geometric constraints as compared to traditional keypoint-based methods. In this work, we introduce an incremental SfM system that, in addition to points, leverages lines and their structured geometric relations. Our technical contributions span the entire pipeline (mapping, triangulation, registration) and we integrate these into a comprehensive end-to-end SfM system that we share as an open-source software with the community. We also present the first analytical method to propagate uncertainties for 3D optimized lines via sensitivity analysis. Experiments show that our system is consistently more robust and accurate compared to the widely used point-based state of the art in SfM – achieving richer maps and more precise camera registrations, especially under challenging conditions. In addition, our uncertainty-aware localization module alone is able to consistently improve over the state of the art under both pointalone and hybrid setups.</p>}},
author = {{Liu, Shaohui and Gao, Yidan and Zhang, Tianyi and Pautrat, Rémi and Schönberger, Johannes L. and Larsson, Viktor and Pollefeys, Marc}},
booktitle = {{Computer Vision – ECCV 2024 - 18th European Conference, Proceedings}},
editor = {{Leonardis, Aleš and Ricci, Elisa and Roth, Stefan and Russakovsky, Olga and Sattler, Torsten and Varol, Gül}},
isbn = {{9783031727634}},
issn = {{1611-3349}},
language = {{eng}},
pages = {{249--269}},
publisher = {{Springer Science and Business Media B.V.}},
series = {{Lecture Notes in Computer Science}},
title = {{Robust Incremental Structure-from-Motion with Hybrid Features}},
url = {{http://dx.doi.org/10.1007/978-3-031-72764-1_15}},
doi = {{10.1007/978-3-031-72764-1_15}},
volume = {{15094 LNCS}},
year = {{2025}},
}