Rank Reduction with Convex Constraints
(2017) In PhD Thesis TFRT-1117- Abstract
- This thesis addresses problems which require low-rank solutions under convex constraints. In particular, the focus lies on model reduction of positive systems, as well as finite dimensional optimization problems that are convex, apart from a low-rank constraint.
Traditional model reduction techniques try to minimize the error between the original and the reduced system. Typically, the resulting reduced models, however, no longer fulfill physically meaningful constraints. This thesis considers the problem of model reduction with internal and external positivity constraints. Both problems are solved by means of balanced truncation. While internal positivity is shown to be preserved by a symmetry property; external positivity... (More) - This thesis addresses problems which require low-rank solutions under convex constraints. In particular, the focus lies on model reduction of positive systems, as well as finite dimensional optimization problems that are convex, apart from a low-rank constraint.
Traditional model reduction techniques try to minimize the error between the original and the reduced system. Typically, the resulting reduced models, however, no longer fulfill physically meaningful constraints. This thesis considers the problem of model reduction with internal and external positivity constraints. Both problems are solved by means of balanced truncation. While internal positivity is shown to be preserved by a symmetry property; external positivity preservation is accomplished by deriving a modified balancing approach based on ellipsoidal cone invariance.
In essence, positivity preserving model reduction attempts to find an infinite dimensional low-rank approximation that preserves nonnegativity, as well as Hankel structure. Due to the non-convexity of the low-rank constraint, this problem is even challenging in a finite dimensional setting. In addition to model reduction, the present work also considers such finite dimensional low-rank optimization problems with convex constraints. These problems frequently appear in applications such as image compression, multivariate linear regression, matrix completion and many more.
The main idea of this thesis is to derive the largest convex minorizers of rank-constrained unitarily invariant norms. These minorizers can be used to construct optimal convex relaxations for the original non-convex problem. Unlike other methods such as nuclear norm regularization, this approach benefits from having verifiable a posterior conditions for which a solution to the convex relaxation and the corresponding non-convex problem coincide. It is shown that this applies to various numerical examples of well-known low-rank optimization problems. In particular, the proposed convex relaxation performs significantly better than nuclear norm regularization. Moreover, it can be observed that a careful choice among the proposed convex relaxations may have a tremendous positive impact on matrix completion.
Computational tractability of the proposed approach is accomplished in two ways. First, the considered relaxations are shown to be representable by semi-definite programs. Second, it is shown how to compute the proximal mappings, for both, the convex relaxations, as well as the non-convex problem. This makes it possible to apply first order method such as so-called Douglas-Rachford splitting. In addition to the convex case, where global convergence of this algorithm is guaranteed, conditions for local convergence in the non-convex setting are presented.
Finally, it is shown that the findings of this thesis also extend to the general class of so-called atomic norms that allow us to cover other non-convex constraints. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/54cb814f-59fe-4bc9-a7ef-773cbcf06889
- author
- Grussler, Christian LU
- supervisor
- opponent
-
- Professor Sepulchre, Rudolf, University of Cambridge
- organization
- publishing date
- 2017
- type
- Thesis
- publication status
- published
- subject
- keywords
- low-rank approximation, model reduction, non-convex optimization, Douglas-Rachford, matrix completion, overlapping norm, k-support norm, atomic norm
- in
- PhD Thesis TFRT-1117
- pages
- 181 pages
- publisher
- Department of Automatic Control, Lund Institute of Technology, Lund University
- defense location
- M:B
- defense date
- 2017-02-03 13:00:00
- ISSN
- 0280-5316
- ISBN
- 978-91-7753-080-0
- 978-91-7753-081-7
- language
- English
- LU publication?
- yes
- additional info
- Series Information: Publication Series: PhD Theses ISSN 0280-5316 Number TFRT-1117
- id
- 54cb814f-59fe-4bc9-a7ef-773cbcf06889
- date added to LUP
- 2017-01-09 03:58:38
- date last changed
- 2022-04-11 12:04:03
@phdthesis{54cb814f-59fe-4bc9-a7ef-773cbcf06889,
abstract = {{This thesis addresses problems which require low-rank solutions under convex constraints. In particular, the focus lies on model reduction of positive systems, as well as finite dimensional optimization problems that are convex, apart from a low-rank constraint. <br/><br/>Traditional model reduction techniques try to minimize the error between the original and the reduced system. Typically, the resulting reduced models, however, no longer fulfill physically meaningful constraints. This thesis considers the problem of model reduction with internal and external positivity constraints. Both problems are solved by means of balanced truncation. While internal positivity is shown to be preserved by a symmetry property; external positivity preservation is accomplished by deriving a modified balancing approach based on ellipsoidal cone invariance.<br/><br/>In essence, positivity preserving model reduction attempts to find an infinite dimensional low-rank approximation that preserves nonnegativity, as well as Hankel structure. Due to the non-convexity of the low-rank constraint, this problem is even challenging in a finite dimensional setting. In addition to model reduction, the present work also considers such finite dimensional low-rank optimization problems with convex constraints. These problems frequently appear in applications such as image compression, multivariate linear regression, matrix completion and many more. <br/><br/>The main idea of this thesis is to derive the largest convex minorizers of rank-constrained unitarily invariant norms. These minorizers can be used to construct optimal convex relaxations for the original non-convex problem. Unlike other methods such as nuclear norm regularization, this approach benefits from having verifiable a posterior conditions for which a solution to the convex relaxation and the corresponding non-convex problem coincide. It is shown that this applies to various numerical examples of well-known low-rank optimization problems. In particular, the proposed convex relaxation performs significantly better than nuclear norm regularization. Moreover, it can be observed that a careful choice among the proposed convex relaxations may have a tremendous positive impact on matrix completion. <br/><br/>Computational tractability of the proposed approach is accomplished in two ways. First, the considered relaxations are shown to be representable by semi-definite programs. Second, it is shown how to compute the proximal mappings, for both, the convex relaxations, as well as the non-convex problem. This makes it possible to apply first order method such as so-called Douglas-Rachford splitting. In addition to the convex case, where global convergence of this algorithm is guaranteed, conditions for local convergence in the non-convex setting are presented. <br/><br/>Finally, it is shown that the findings of this thesis also extend to the general class of so-called atomic norms that allow us to cover other non-convex constraints.}},
author = {{Grussler, Christian}},
isbn = {{978-91-7753-080-0}},
issn = {{0280-5316}},
keywords = {{low-rank approximation; model reduction; non-convex optimization; Douglas-Rachford; matrix completion; overlapping norm; k-support norm; atomic norm}},
language = {{eng}},
publisher = {{Department of Automatic Control, Lund Institute of Technology, Lund University}},
school = {{Lund University}},
series = {{PhD Thesis TFRT-1117}},
title = {{Rank Reduction with Convex Constraints}},
url = {{https://lup.lub.lu.se/search/files/19595129/Thesis.pdf}},
year = {{2017}},
}