Hyperparameter Selection for Group-Sparse Regression: A Probabilistic Approach
Kronvall, Ted LU and Jakobsson, Andreas LU
(2018)
In Signal Processing
151.
p.107-118
- Abstract
- This work analyzes the effects on support recovery for different choices of the hyper- or regularization parameter in LASSO-like sparse and group-sparse regression problems. The hyperparameter implicitly selects the model order of the solution, and is typically set using cross-validation (CV). This may be computationally prohibitive for large-scale problems, and also often overestimates the model order, as CV optimizes for prediction error rather than support recovery. In this work, we propose a probabilistic approach to select the hyperparameter, by quantifying the type I error (false positive rate) using extreme value analysis. From Monte Carlo simulations, one may draw inference on the upper tail of the distribution of the spurious... (More)
- This work analyzes the effects on support recovery for different choices of the hyper- or regularization parameter in LASSO-like sparse and group-sparse regression problems. The hyperparameter implicitly selects the model order of the solution, and is typically set using cross-validation (CV). This may be computationally prohibitive for large-scale problems, and also often overestimates the model order, as CV optimizes for prediction error rather than support recovery. In this work, we propose a probabilistic approach to select the hyperparameter, by quantifying the type I error (false positive rate) using extreme value analysis. From Monte Carlo simulations, one may draw inference on the upper tail of the distribution of the spurious parameter estimates, and the regularization level may be selected for a specified false positive rate. By solving the e group-LASSO problem, the choice of hyperparameter becomes independent of the noise variance. Furthermore, the effects on the false positive rate caused by collinearity in the dictionary is discussed, including ways of circumventing them. The proposed method is compared to other hyperparameter-selection methods in terms of support recovery, false positive rate, false negative rate, and computational complexity. Simulated data illustrate how the proposed method outperforms CV and comparable methods in both computational complexity and support recovery. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/9d6bb9cb-bd2f-4c47-a5b9-195e3b9b8b8e
- author
- Kronvall, Ted
LU
and Jakobsson, Andreas
LU
- organization
- publishing date
- 2018
- type
- Contribution to journal
- publication status
- published
- subject
- in
- Signal Processing
- volume
- 151
- pages
- 107 - 118
- publisher
- Elsevier
- external identifiers
-
- scopus:85047090258
- ISSN
- 0165-1684
- DOI
- 10.1016/j.sigpro.2018.04.021
- project
- eSSENCE@LU 4:2 - Efficient data acquisition and analyses for modern multidimensional spectroscopy"
- language
- English
- LU publication?
- yes
- id
- 9d6bb9cb-bd2f-4c47-a5b9-195e3b9b8b8e
- date added to LUP
- 2017-10-05 14:04:11
- date last changed
- 2022-04-25 02:58:56
@article{9d6bb9cb-bd2f-4c47-a5b9-195e3b9b8b8e,
abstract = {{This work analyzes the effects on support recovery for different choices of the hyper- or regularization parameter in LASSO-like sparse and group-sparse regression problems. The hyperparameter implicitly selects the model order of the solution, and is typically set using cross-validation (CV). This may be computationally prohibitive for large-scale problems, and also often overestimates the model order, as CV optimizes for prediction error rather than support recovery. In this work, we propose a probabilistic approach to select the hyperparameter, by quantifying the type I error (false positive rate) using extreme value analysis. From Monte Carlo simulations, one may draw inference on the upper tail of the distribution of the spurious parameter estimates, and the regularization level may be selected for a specified false positive rate. By solving the e group-LASSO problem, the choice of hyperparameter becomes independent of the noise variance. Furthermore, the effects on the false positive rate caused by collinearity in the dictionary is discussed, including ways of circumventing them. The proposed method is compared to other hyperparameter-selection methods in terms of support recovery, false positive rate, false negative rate, and computational complexity. Simulated data illustrate how the proposed method outperforms CV and comparable methods in both computational complexity and support recovery.}},
author = {{Kronvall, Ted and Jakobsson, Andreas}},
issn = {{0165-1684}},
language = {{eng}},
pages = {{107--118}},
publisher = {{Elsevier}},
series = {{Signal Processing}},
title = {{Hyperparameter Selection for Group-Sparse Regression: A Probabilistic Approach}},
url = {{https://lup.lub.lu.se/search/files/32747751/manuscript_singlecolumn.pdf}},
doi = {{10.1016/j.sigpro.2018.04.021}},
volume = {{151}},
year = {{2018}},
}