Differentiable fixed-rank regularisation using bilinear parameterisation
Örnhag, Marcus Valtonen LU ; Olsson, Carl LU and Heyden, Anders LU
(2020)
30th British Machine Vision Conference, BMVC 2019
- Abstract
Low rank structures are present in many applications of computer vision and machine learning. A popular approach consists of explicitly parameterising the set or matrices with sought rank, leading to a bilinear factorisation, reducing the problem to find the bilinear factors. While such an approach can be efficiently implemented using second-order methods, such as Levenberg-Marquardt (LM) or Variable Projection (VarPro), it suffers from the presence of local minima, which makes theoretical optimality guarantees hard to derive. Another approach is to penalise non-zero singular values to enforce a low-rank structure. In certain cases, global optimality guarantees are known; however, such methods often lead to non-differentiable (and even... (More)
Low rank structures are present in many applications of computer vision and machine learning. A popular approach consists of explicitly parameterising the set or matrices with sought rank, leading to a bilinear factorisation, reducing the problem to find the bilinear factors. While such an approach can be efficiently implemented using second-order methods, such as Levenberg-Marquardt (LM) or Variable Projection (VarPro), it suffers from the presence of local minima, which makes theoretical optimality guarantees hard to derive. Another approach is to penalise non-zero singular values to enforce a low-rank structure. In certain cases, global optimality guarantees are known; however, such methods often lead to non-differentiable (and even discontinuous) objectives, for which it is necessary to use subgradient methods and splitting schemes. If the objective is complex, such as in structure from motion, the convergence rates for such methods can be very slow. In this paper we show how optimality guarantees can be lifted to methods that employ bilinear parameterisation when the sought rank is known. Using this approach the best of two worlds are combined: optimality guarantees and superior convergence speeds. We compare the proposed method to state-of-the-art solvers for prior-free non-rigid structure from motion.
(Less)
- author
- Örnhag, Marcus Valtonen
LU
; Olsson, Carl
LU
and Heyden, Anders
LU
- organization
- publishing date
- 2020
- type
- Contribution to conference
- publication status
- published
- subject
- conference name
- 30th British Machine Vision Conference, BMVC 2019
- conference location
- Cardiff, United Kingdom
- conference dates
- 2019-09-09 - 2019-09-12
- external identifiers
-
- scopus:85087334181
- language
- English
- LU publication?
- yes
- id
- 917a4f42-940e-4b27-9f9f-30250c39c4a5
- alternative location
- https://bmvc2019.org/wp-content/uploads/papers/0199-paper.pdf
- date added to LUP
- 2020-07-17 13:29:40
- date last changed
- 2023-12-04 17:20:36
@misc{917a4f42-940e-4b27-9f9f-30250c39c4a5,
abstract = {{<p>Low rank structures are present in many applications of computer vision and machine learning. A popular approach consists of explicitly parameterising the set or matrices with sought rank, leading to a bilinear factorisation, reducing the problem to find the bilinear factors. While such an approach can be efficiently implemented using second-order methods, such as Levenberg-Marquardt (LM) or Variable Projection (VarPro), it suffers from the presence of local minima, which makes theoretical optimality guarantees hard to derive. Another approach is to penalise non-zero singular values to enforce a low-rank structure. In certain cases, global optimality guarantees are known; however, such methods often lead to non-differentiable (and even discontinuous) objectives, for which it is necessary to use subgradient methods and splitting schemes. If the objective is complex, such as in structure from motion, the convergence rates for such methods can be very slow. In this paper we show how optimality guarantees can be lifted to methods that employ bilinear parameterisation when the sought rank is known. Using this approach the best of two worlds are combined: optimality guarantees and superior convergence speeds. We compare the proposed method to state-of-the-art solvers for prior-free non-rigid structure from motion.</p>}},
author = {{Örnhag, Marcus Valtonen and Olsson, Carl and Heyden, Anders}},
language = {{eng}},
title = {{Differentiable fixed-rank regularisation using bilinear parameterisation}},
url = {{https://bmvc2019.org/wp-content/uploads/papers/0199-paper.pdf}},
year = {{2020}},
}