False discoveries occur early on the lasso path
(2017) In Annals of Statistics 45(5). p.2133-2150- Abstract
In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity-meaning that the fraction of variables with a nonvanishing effect tends to a constant, however small-this cannot really be the case, even when the design variables are stochastically independent. We demonstrate that true features and null features are always interspersed on the Lasso path, and that this phenomenon occurs no matter how strong the effect sizes are. We derive a sharp asymptotic trade-off between false and true positive rates or, equivalently, between... (More)
In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity-meaning that the fraction of variables with a nonvanishing effect tends to a constant, however small-this cannot really be the case, even when the design variables are stochastically independent. We demonstrate that true features and null features are always interspersed on the Lasso path, and that this phenomenon occurs no matter how strong the effect sizes are. We derive a sharp asymptotic trade-off between false and true positive rates or, equivalently, between measures of type I and type II errors along the Lasso path. This trade-off states that if we ever want to achieve a type II error (false negative rate) under a critical value, then anywhere on the Lasso path the type I error (false positive rate) will need to exceed a given threshold so that we can never have both errors at a low level at the same time. Our analysis uses tools from approximate message passing (AMP) theory as well as novel elements to deal with a possibly adaptive selection of the Lasso regularizing parameter.
(Less)
- author
- Su, Weijie ; Bogdan, Malgorzata LU and Candès, Emmanuel
- publishing date
- 2017-10
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Adaptive selection of parameters, Approximate message passing (AMP), False discovery rate, False negative rate, Lasso, Lasso path, Power
- in
- Annals of Statistics
- volume
- 45
- issue
- 5
- pages
- 18 pages
- publisher
- Institute of Mathematical Statistics
- external identifiers
-
- scopus:85034859569
- ISSN
- 0090-5364
- DOI
- 10.1214/16-AOS1521
- language
- English
- LU publication?
- no
- additional info
- Funding Information: Received June 2016; revised September 2016. 1Supported in part by NSF Grant CCF-0963835. 2Supported in part by the European Union’s 7th Framework Programme for research, technological development and demonstration under Grant Agreement No. 602552 and co-financed by the Polish Ministry of Science and Higher Education under Grant Agreement 2932/7.PR/2013/2. 3Supported in part by NSF Grant CCF-0963835 and by the Math + X Award from the Simons Foundation. MSC2010 subject classifications. Primary 62F03; secondary 62J07, 62J05. Key words and phrases. Lasso, Lasso path, false discovery rate, false negative rate, power, approximate message passing (AMP), adaptive selection of parameters. Publisher Copyright: © Institute of Mathematical Statistics, 2017.
- id
- f4099ecf-8431-4c79-899a-9b86b5f1da81
- date added to LUP
- 2023-12-08 09:21:38
- date last changed
- 2023-12-11 08:49:42
@article{f4099ecf-8431-4c79-899a-9b86b5f1da81,
abstract = {{<p>In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity-meaning that the fraction of variables with a nonvanishing effect tends to a constant, however small-this cannot really be the case, even when the design variables are stochastically independent. We demonstrate that true features and null features are always interspersed on the Lasso path, and that this phenomenon occurs no matter how strong the effect sizes are. We derive a sharp asymptotic trade-off between false and true positive rates or, equivalently, between measures of type I and type II errors along the Lasso path. This trade-off states that if we ever want to achieve a type II error (false negative rate) under a critical value, then anywhere on the Lasso path the type I error (false positive rate) will need to exceed a given threshold so that we can never have both errors at a low level at the same time. Our analysis uses tools from approximate message passing (AMP) theory as well as novel elements to deal with a possibly adaptive selection of the Lasso regularizing parameter.</p>}},
author = {{Su, Weijie and Bogdan, Malgorzata and Candès, Emmanuel}},
issn = {{0090-5364}},
keywords = {{Adaptive selection of parameters; Approximate message passing (AMP); False discovery rate; False negative rate; Lasso; Lasso path; Power}},
language = {{eng}},
number = {{5}},
pages = {{2133--2150}},
publisher = {{Institute of Mathematical Statistics}},
series = {{Annals of Statistics}},
title = {{False discoveries occur early on the lasso path}},
url = {{http://dx.doi.org/10.1214/16-AOS1521}},
doi = {{10.1214/16-AOS1521}},
volume = {{45}},
year = {{2017}},
}