Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

The Hessian Screening Rule

Larsson, Johan LU orcid and Wallin, Jonas LU (2022) 36th Conference on Neural Information Processing Systems, NeurIPS 2022 In Advances in Neural Information Processing Systems 35. p.25404-25421
Abstract

Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `1-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.

Please use this url to cite or link to this publication:
author
and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
Advances in Neural Information Processing Systems
series title
Advances in Neural Information Processing Systems
editor
Koyejo, S. ; Mohamed, S. ; Agarwal, A. ; Belgrave, D. ; Cho, K. and Oh, A.
volume
35
pages
25404 - 25421
publisher
Curran Associates, Inc
conference name
36th Conference on Neural Information Processing Systems, NeurIPS 2022
conference location
New Orleans, United States
conference dates
2022-11-28 - 2022-12-09
external identifiers
  • scopus:85163193107
ISSN
1049-5258
ISBN
9781713871088
project
Optimization and Algorithms for Sparse Regression
language
English
LU publication?
yes
additional info
Funding Information: We would like to thank Małgorzata Bogdan for valuable comments. This work was funded by the Swedish Research Council through grant agreement no. 2020-05081 and no. 2018-01726. The computations were enabled by resources provided by LUNARC. The results shown here are in part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga. Publisher Copyright: © 2022 Neural information processing systems foundation. All rights reserved.
id
8303a35c-a830-4ed5-9ea4-1f33d6c0fb66
date added to LUP
2023-09-14 11:56:48
date last changed
2023-12-07 09:00:12
@inproceedings{8303a35c-a830-4ed5-9ea4-1f33d6c0fb66,
  abstract     = {{<p>Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for `<sub>1</sub>-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.</p>}},
  author       = {{Larsson, Johan and Wallin, Jonas}},
  booktitle    = {{Advances in Neural Information Processing Systems}},
  editor       = {{Koyejo, S. and Mohamed, S. and Agarwal, A. and Belgrave, D. and Cho, K. and Oh, A.}},
  isbn         = {{9781713871088}},
  issn         = {{1049-5258}},
  language     = {{eng}},
  month        = {{12}},
  pages        = {{25404--25421}},
  publisher    = {{Curran Associates, Inc}},
  series       = {{Advances in Neural Information Processing Systems}},
  title        = {{The Hessian Screening Rule}},
  volume       = {{35}},
  year         = {{2022}},
}