Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Unveiling low-dimensional patterns induced by convex non-differentiable regularizers

Hejný, Ivan LU ; Wallin, Jonas LU ; Bogdan, Małgorzata LU and Kos, Michał LU orcid (2025) In Annals of the Institute of Statistical Mathematics
Abstract

This paper explores the asymptotic distributions of low-dimensional patterns in linear regression with regularizers such as Lasso, Elastic Net, Generalized Lasso, and SLOPE, as the number of observations n grows and the penalty increases at rate n. While the asymptotic distribution of rescaled estimation errors is well-understood, convergence of patterns lacks proof in the literature, even for Lasso. We provide a proof using the Hausdorff distance for subdifferentials. We also derive the limiting probability of recovering the true model pattern, which approaches 1 when the penalty scaling diverges and the regularizer-specific asymptotic irrepresentability condition is satisfied. We propose two-step procedures that asymptotically recover... (More)

This paper explores the asymptotic distributions of low-dimensional patterns in linear regression with regularizers such as Lasso, Elastic Net, Generalized Lasso, and SLOPE, as the number of observations n grows and the penalty increases at rate n. While the asymptotic distribution of rescaled estimation errors is well-understood, convergence of patterns lacks proof in the literature, even for Lasso. We provide a proof using the Hausdorff distance for subdifferentials. We also derive the limiting probability of recovering the true model pattern, which approaches 1 when the penalty scaling diverges and the regularizer-specific asymptotic irrepresentability condition is satisfied. We propose two-step procedures that asymptotically recover model patterns, regardless of the irrepresentability condition. Our theory shows that Fused Lasso cannot reliably recover its clustering pattern for independent regressors, but this can be resolved by concavifying its penalty coefficients. Simulation studies compare the asymptotic properties of Lasso, Fused Lasso, and SLOPE.

(Less)
Please use this url to cite or link to this publication:
author
; ; and
organization
publishing date
type
Contribution to journal
publication status
epub
subject
keywords
Hausdorff distance, Irrepresentability condition, Pattern convergence, Pattern recovery, Regularization, Subdifferential
in
Annals of the Institute of Statistical Mathematics
publisher
Springer
external identifiers
  • scopus:105020867712
ISSN
0020-3157
DOI
10.1007/s10463-025-00957-6
language
English
LU publication?
yes
additional info
Publisher Copyright: © The Institute of Statistical Mathematics, Tokyo 2025.
id
9a7358eb-6d45-4a97-b053-d2c38aee8bcb
date added to LUP
2026-01-14 09:45:29
date last changed
2026-01-14 09:46:28
@article{9a7358eb-6d45-4a97-b053-d2c38aee8bcb,
  abstract     = {{<p>This paper explores the asymptotic distributions of low-dimensional patterns in linear regression with regularizers such as Lasso, Elastic Net, Generalized Lasso, and SLOPE, as the number of observations n grows and the penalty increases at rate n. While the asymptotic distribution of rescaled estimation errors is well-understood, convergence of patterns lacks proof in the literature, even for Lasso. We provide a proof using the Hausdorff distance for subdifferentials. We also derive the limiting probability of recovering the true model pattern, which approaches 1 when the penalty scaling diverges and the regularizer-specific asymptotic irrepresentability condition is satisfied. We propose two-step procedures that asymptotically recover model patterns, regardless of the irrepresentability condition. Our theory shows that Fused Lasso cannot reliably recover its clustering pattern for independent regressors, but this can be resolved by concavifying its penalty coefficients. Simulation studies compare the asymptotic properties of Lasso, Fused Lasso, and SLOPE.</p>}},
  author       = {{Hejný, Ivan and Wallin, Jonas and Bogdan, Małgorzata and Kos, Michał}},
  issn         = {{0020-3157}},
  keywords     = {{Hausdorff distance; Irrepresentability condition; Pattern convergence; Pattern recovery; Regularization; Subdifferential}},
  language     = {{eng}},
  publisher    = {{Springer}},
  series       = {{Annals of the Institute of Statistical Mathematics}},
  title        = {{Unveiling low-dimensional patterns induced by convex non-differentiable regularizers}},
  url          = {{http://dx.doi.org/10.1007/s10463-025-00957-6}},
  doi          = {{10.1007/s10463-025-00957-6}},
  year         = {{2025}},
}