Feature selection in jump models
Nystrup, Peter LU
; Kolm, Petter N.
and Lindström, Erik
LU
(2021)
In Expert Systems with Applications
184.
- Abstract
- Jump models switch infrequently between states to fit a sequence of data while taking the ordering of the data into account We propose a new framework for joint feature selection, parameter and state-sequence estimation in jump models. Feature selection is necessary in high-dimensional settings where the number of features is large compared to the number of observations and the underlying states differ only with respect to a subset of the features. We develop and implement a coordinate descent algorithm that alternates between selecting the features and estimating the model parameters and state sequence, which scales to large data sets with large numbers of (noisy) features. We demonstrate the usefulness of the proposed framework by... (More)
- Jump models switch infrequently between states to fit a sequence of data while taking the ordering of the data into account We propose a new framework for joint feature selection, parameter and state-sequence estimation in jump models. Feature selection is necessary in high-dimensional settings where the number of features is large compared to the number of observations and the underlying states differ only with respect to a subset of the features. We develop and implement a coordinate descent algorithm that alternates between selecting the features and estimating the model parameters and state sequence, which scales to large data sets with large numbers of (noisy) features. We demonstrate the usefulness of the proposed framework by comparing it with a number of other methods on both simulated and real data in the form of financial returns, protein sequences, and text. By leveraging information embedded in the ordering of the data, the resulting sparse jump model outperforms all other methods considered and is remarkably robust to noise. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/c4d484c9-ebb4-4f80-9f90-b7580e313164
- author
- Nystrup, Peter
LU
; Kolm, Petter N.
and Lindström, Erik
LU
- organization
- publishing date
- 2021-12-01
- type
- Contribution to journal
- publication status
- published
- subject
- in
- Expert Systems with Applications
- volume
- 184
- article number
- 115558
- pages
- 14 pages
- publisher
- Elsevier
- external identifiers
-
- scopus:85110526253
- ISSN
- 0957-4174
- DOI
- 10.1016/j.eswa.2021.115558
- language
- English
- LU publication?
- yes
- id
- c4d484c9-ebb4-4f80-9f90-b7580e313164
- alternative location
- https://linkinghub.elsevier.com/retrieve/pii/S0957417421009647
- date added to LUP
- 2021-09-09 14:57:19
- date last changed
- 2023-11-07 11:42:38
@article{c4d484c9-ebb4-4f80-9f90-b7580e313164,
abstract = {{Jump models switch infrequently between states to fit a sequence of data while taking the ordering of the data into account We propose a new framework for joint feature selection, parameter and state-sequence estimation in jump models. Feature selection is necessary in high-dimensional settings where the number of features is large compared to the number of observations and the underlying states differ only with respect to a subset of the features. We develop and implement a coordinate descent algorithm that alternates between selecting the features and estimating the model parameters and state sequence, which scales to large data sets with large numbers of (noisy) features. We demonstrate the usefulness of the proposed framework by comparing it with a number of other methods on both simulated and real data in the form of financial returns, protein sequences, and text. By leveraging information embedded in the ordering of the data, the resulting sparse jump model outperforms all other methods considered and is remarkably robust to noise.}},
author = {{Nystrup, Peter and Kolm, Petter N. and Lindström, Erik}},
issn = {{0957-4174}},
language = {{eng}},
month = {{12}},
publisher = {{Elsevier}},
series = {{Expert Systems with Applications}},
title = {{Feature selection in jump models}},
url = {{http://dx.doi.org/10.1016/j.eswa.2021.115558}},
doi = {{10.1016/j.eswa.2021.115558}},
volume = {{184}},
year = {{2021}},
}