Advanced

Hyperparameter-selection for sparse regression : A probablistic approach

Kronvall, Ted LU and Jakobsson, Andreas LU (2018) 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 2017-October. p.853-857
Abstract

The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter... (More)

The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter becomes almost independent of the noise variance. Simulation examples illustrate how the proposed method outperforms both cross-validation and the Bayesian Information Criterion in terms of computational complexity and support recovery.

(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
extreme value distribution, LASSO, model order estimation, regularization, sparse estimation
host publication
Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
volume
2017-October
pages
5 pages
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
conference name
51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
conference location
Pacific Grove, United States
conference dates
2017-10-29 - 2017-11-01
external identifiers
  • scopus:85050925756
ISBN
9781538618233
DOI
10.1109/ACSSC.2017.8335469
language
English
LU publication?
yes
id
aa42b4ae-97d5-42a1-bded-50cb3989fb98
date added to LUP
2018-09-17 13:12:56
date last changed
2019-04-28 17:49:33
@inproceedings{aa42b4ae-97d5-42a1-bded-50cb3989fb98,
  abstract     = {<p>The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter becomes almost independent of the noise variance. Simulation examples illustrate how the proposed method outperforms both cross-validation and the Bayesian Information Criterion in terms of computational complexity and support recovery.</p>},
  author       = {Kronvall, Ted and Jakobsson, Andreas},
  isbn         = {9781538618233},
  keyword      = {extreme value distribution,LASSO,model order estimation,regularization,sparse estimation},
  language     = {eng},
  location     = {Pacific Grove, United States},
  month        = {04},
  pages        = {853--857},
  publisher    = {IEEE - Institute of Electrical and Electronics Engineers Inc.},
  title        = {Hyperparameter-selection for sparse regression : A probablistic approach},
  url          = {http://dx.doi.org/10.1109/ACSSC.2017.8335469},
  volume       = {2017-October},
  year         = {2018},
}