Advanced

Large displacement 3D scene flow with occlusion reasoning

Zanfir, Andrei and Sminchisescu, Cristian LU (2016) 15th IEEE International Conference on Computer Vision, ICCV 2015 In Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015 11-18-December-2015. p.4417-4425
Abstract

The emergence of modern, affordable and accurate RGB-D sensors increases the need for single view approaches to estimate 3-dimensional motion, also known as scene flow. In this paper we propose a coarse-to-fine, dense, correspondence-based scene flow formulation that relies on explicit geometric reasoning to account for the effects of large displacements and to model occlusion. Our methodology enforces local motion rigidity at the level of the 3d point cloud without explicitly smoothing the parameters of adjacent neighborhoods. By integrating all geometric and photometric components in a single, consistent, occlusion-aware energy model, defined over overlapping, image-adaptive neighborhoods, our method can process fast motions and large... (More)

The emergence of modern, affordable and accurate RGB-D sensors increases the need for single view approaches to estimate 3-dimensional motion, also known as scene flow. In this paper we propose a coarse-to-fine, dense, correspondence-based scene flow formulation that relies on explicit geometric reasoning to account for the effects of large displacements and to model occlusion. Our methodology enforces local motion rigidity at the level of the 3d point cloud without explicitly smoothing the parameters of adjacent neighborhoods. By integrating all geometric and photometric components in a single, consistent, occlusion-aware energy model, defined over overlapping, image-adaptive neighborhoods, our method can process fast motions and large occlusions areas, as present in challenging datasets like the MPI Sintel Flow Dataset, recently augmented with depth information. By explicitly modeling large displacements and occlusion, we can handle difficult sequences which cannot be currently processed by state of the art scene flow methods. We also show that by integrating depth information into the model, we can obtain correspondence fields with improved spatial support and sharper boundaries compared to the state of the art, large-displacement optical flow methods.

(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
in
Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015
volume
11-18-December-2015
pages
9 pages
publisher
Institute of Electrical and Electronics Engineers Inc.
conference name
15th IEEE International Conference on Computer Vision, ICCV 2015
external identifiers
  • scopus:84973855972
ISBN
9781467383912
DOI
10.1109/ICCV.2015.502
language
English
LU publication?
yes
id
d22596d7-d5db-4c0d-b3dd-60b7094af553
date added to LUP
2017-02-13 14:10:26
date last changed
2017-02-13 14:10:26
@inproceedings{d22596d7-d5db-4c0d-b3dd-60b7094af553,
  abstract     = {<p>The emergence of modern, affordable and accurate RGB-D sensors increases the need for single view approaches to estimate 3-dimensional motion, also known as scene flow. In this paper we propose a coarse-to-fine, dense, correspondence-based scene flow formulation that relies on explicit geometric reasoning to account for the effects of large displacements and to model occlusion. Our methodology enforces local motion rigidity at the level of the 3d point cloud without explicitly smoothing the parameters of adjacent neighborhoods. By integrating all geometric and photometric components in a single, consistent, occlusion-aware energy model, defined over overlapping, image-adaptive neighborhoods, our method can process fast motions and large occlusions areas, as present in challenging datasets like the MPI Sintel Flow Dataset, recently augmented with depth information. By explicitly modeling large displacements and occlusion, we can handle difficult sequences which cannot be currently processed by state of the art scene flow methods. We also show that by integrating depth information into the model, we can obtain correspondence fields with improved spatial support and sharper boundaries compared to the state of the art, large-displacement optical flow methods.</p>},
  author       = {Zanfir, Andrei and Sminchisescu, Cristian},
  booktitle    = {Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015},
  isbn         = {9781467383912},
  language     = {eng},
  month        = {02},
  pages        = {4417--4425},
  publisher    = {Institute of Electrical and Electronics Engineers Inc.},
  title        = {Large displacement 3D scene flow with occlusion reasoning},
  url          = {http://dx.doi.org/10.1109/ICCV.2015.502},
  volume       = {11-18-December-2015},
  year         = {2016},
}