The Hessian Screening Rule
(2022) 36th Conference on Neural Information Processing Systems, NeurIPS 2022 In Advances in Neural Information Processing Systems 35. p.25404-25421- Abstract
Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `1-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/8303a35c-a830-4ed5-9ea4-1f33d6c0fb66
- author
- Larsson, Johan LU and Wallin, Jonas LU
- organization
- publishing date
- 2022-12-06
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- host publication
- Advances in Neural Information Processing Systems
- series title
- Advances in Neural Information Processing Systems
- editor
- Koyejo, S. ; Mohamed, S. ; Agarwal, A. ; Belgrave, D. ; Cho, K. and Oh, A.
- volume
- 35
- pages
- 25404 - 25421
- publisher
- Curran Associates, Inc
- conference name
- 36th Conference on Neural Information Processing Systems, NeurIPS 2022
- conference location
- New Orleans, United States
- conference dates
- 2022-11-28 - 2022-12-09
- external identifiers
-
- scopus:85163193107
- ISSN
- 1049-5258
- ISBN
- 9781713871088
- project
- Optimization and Algorithms for Sparse Regression
- language
- English
- LU publication?
- yes
- additional info
- Funding Information: We would like to thank Małgorzata Bogdan for valuable comments. This work was funded by the Swedish Research Council through grant agreement no. 2020-05081 and no. 2018-01726. The computations were enabled by resources provided by LUNARC. The results shown here are in part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga. Publisher Copyright: © 2022 Neural information processing systems foundation. All rights reserved.
- id
- 8303a35c-a830-4ed5-9ea4-1f33d6c0fb66
- date added to LUP
- 2023-09-14 11:56:48
- date last changed
- 2023-12-07 09:00:12
@inproceedings{8303a35c-a830-4ed5-9ea4-1f33d6c0fb66, abstract = {{<p>Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `<sub>1</sub>-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.</p>}}, author = {{Larsson, Johan and Wallin, Jonas}}, booktitle = {{Advances in Neural Information Processing Systems}}, editor = {{Koyejo, S. and Mohamed, S. and Agarwal, A. and Belgrave, D. and Cho, K. and Oh, A.}}, isbn = {{9781713871088}}, issn = {{1049-5258}}, language = {{eng}}, month = {{12}}, pages = {{25404--25421}}, publisher = {{Curran Associates, Inc}}, series = {{Advances in Neural Information Processing Systems}}, title = {{The Hessian Screening Rule}}, volume = {{35}}, year = {{2022}}, }