Hyperparameter-selection for sparse regression : A probablistic approach
(2018) 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 2017-October. p.853-857- Abstract
The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter... (More)
The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter becomes almost independent of the noise variance. Simulation examples illustrate how the proposed method outperforms both cross-validation and the Bayesian Information Criterion in terms of computational complexity and support recovery.
(Less)
- author
- Kronvall, Ted LU and Jakobsson, Andreas LU
- organization
- publishing date
- 2018-04-10
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- keywords
- extreme value distribution, LASSO, model order estimation, regularization, sparse estimation
- host publication
- Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
- volume
- 2017-October
- pages
- 5 pages
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- conference name
- 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
- conference location
- Pacific Grove, United States
- conference dates
- 2017-10-29 - 2017-11-01
- external identifiers
-
- scopus:85050925756
- ISBN
- 9781538618233
- DOI
- 10.1109/ACSSC.2017.8335469
- project
- eSSENCE@LU 4:2 - Efficient data acquisition and analyses for modern multidimensional spectroscopy"
- language
- English
- LU publication?
- yes
- id
- aa42b4ae-97d5-42a1-bded-50cb3989fb98
- date added to LUP
- 2018-09-17 13:12:56
- date last changed
- 2022-01-31 05:20:55
@inproceedings{aa42b4ae-97d5-42a1-bded-50cb3989fb98, abstract = {{<p>The choice of hyperparameter(s) notably affects the support recovery in LASSO-like sparse regression problems, acting as an implicit model order selection. Parameters are typically selected using cross-validation or various ad hoc approaches. These often overestimates the resulting model order, aiming to minimize the prediction error rather than maximizing the support recovery. In this work, we propose a probabilistic approach to selecting hyperparameters in order to maximize the support recovery, quantifying the type I error (false positive rate) using extreme value analysis, such that the regularization level is selected as an appropriate quantile. By instead solving the scaled LASSO problem, the proposed choice of hyperparameter becomes almost independent of the noise variance. Simulation examples illustrate how the proposed method outperforms both cross-validation and the Bayesian Information Criterion in terms of computational complexity and support recovery.</p>}}, author = {{Kronvall, Ted and Jakobsson, Andreas}}, booktitle = {{Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017}}, isbn = {{9781538618233}}, keywords = {{extreme value distribution; LASSO; model order estimation; regularization; sparse estimation}}, language = {{eng}}, month = {{04}}, pages = {{853--857}}, publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}}, title = {{Hyperparameter-selection for sparse regression : A probablistic approach}}, url = {{http://dx.doi.org/10.1109/ACSSC.2017.8335469}}, doi = {{10.1109/ACSSC.2017.8335469}}, volume = {{2017-October}}, year = {{2018}}, }