Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Coordinate Descent for SLOPE

Larsson, Johan LU orcid ; Klopfenstein, Quentin ; Massias, Mathurin and Wallin, Jonas LU (2023) 26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023 In Proceedings of Machine Learning Research 206. p.4802-4821
Abstract

The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty... (More)

The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we demonstrate our method's performance against a long list of competing algorithms.

(Less)
Please use this url to cite or link to this publication:
author
; ; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
Proceedings of the 26th international conference on artificial intelligence and statistics
series title
Proceedings of Machine Learning Research
editor
Ruiz, Francisco ; Dy, Jennifer and van de Meent, Jan-Willem
volume
206
pages
20 pages
conference name
26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023
conference location
Valencia, Spain
conference dates
2023-04-25 - 2023-04-27
external identifiers
  • scopus:85165164923
  • scopus:85165164923
project
Optimization and Algorithms for Sparse Regression
language
English
LU publication?
yes
id
d8bb60fa-3ca6-4d89-88c7-830f57d5707b
alternative location
https://proceedings.mlr.press/v206/larsson23a/larsson23a.pdf
date added to LUP
2023-06-15 13:24:58
date last changed
2023-11-22 22:29:35
@inproceedings{d8bb60fa-3ca6-4d89-88c7-830f57d5707b,
  abstract     = {{<p>The lasso is the most famous sparse regression and feature selection method. One reason for its popularity is the speed at which the underlying optimization problem can be solved. Sorted L-One Penalized Estimation (SLOPE) is a generalization of the lasso with appealing statistical properties. In spite of this, the method has not yet reached widespread interest. A major reason for this is that current software packages that fit SLOPE rely on algorithms that perform poorly in high dimensions. To tackle this issue, we propose a new fast algorithm to solve the SLOPE optimization problem, which combines proximal gradient descent and proximal coordinate descent steps. We provide new results on the directional derivative of the SLOPE penalty and its related SLOPE thresholding operator, as well as provide convergence guarantees for our proposed solver. In extensive benchmarks on simulated and real data, we demonstrate our method's performance against a long list of competing algorithms.</p>}},
  author       = {{Larsson, Johan and Klopfenstein, Quentin and Massias, Mathurin and Wallin, Jonas}},
  booktitle    = {{Proceedings of the 26th international conference on artificial intelligence and statistics}},
  editor       = {{Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}},
  language     = {{eng}},
  pages        = {{4802--4821}},
  series       = {{Proceedings of Machine Learning Research}},
  title        = {{Coordinate Descent for SLOPE}},
  url          = {{https://proceedings.mlr.press/v206/larsson23a/larsson23a.pdf}},
  volume       = {{206}},
  year         = {{2023}},
}