Low Bias Local Intrinsic Dimension Estimation from Expected Simplex Skewness
(2015) In IEEE Transactions on Pattern Analysis and Machine Intelligence 37(1). p.196-202- Abstract
- In exploratory high-dimensional data analysis, local intrinsic dimension estimation can sometimes be used in order to discriminate between data sets sampled from different low-dimensional structures. Global intrinsic dimension estimators can in many cases be adapted to local estimation, but this leads to problems with high negative bias or high variance. We introduce a method that exploits the curse/blessing of dimensionality and produces local intrinsic dimension estimators that have very low bias, even in cases where the intrinsic dimension is higher than the number of data points, in combination with relatively low variance. We show that our estimators have a very good ability to classify local data sets by their dimension compared to... (More)
- In exploratory high-dimensional data analysis, local intrinsic dimension estimation can sometimes be used in order to discriminate between data sets sampled from different low-dimensional structures. Global intrinsic dimension estimators can in many cases be adapted to local estimation, but this leads to problems with high negative bias or high variance. We introduce a method that exploits the curse/blessing of dimensionality and produces local intrinsic dimension estimators that have very low bias, even in cases where the intrinsic dimension is higher than the number of data points, in combination with relatively low variance. We show that our estimators have a very good ability to classify local data sets by their dimension compared to other local intrinsic dimension estimators; furthermore we provide examples showing the usefulness of local intrinsic dimension estimation in general and our method in particular for stratification of real data sets. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/4622255
- author
- Johnsson, Kerstin LU ; Soneson, Charlotte and Fontes, Magnus LU
- organization
- publishing date
- 2015
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Eigenvalues and eigenfunctions, Calibration, Estimation, Dimension, Manifolds
- in
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- volume
- 37
- issue
- 1
- pages
- 196 - 202
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- external identifiers
-
- wos:000346970600017
- scopus:84916930372
- pmid:26353219
- ISSN
- 1939-3539
- DOI
- 10.1109/TPAMI.2014.2343220
- language
- English
- LU publication?
- yes
- id
- 4c17cb4c-ff2d-48ef-b2f5-c0930c6aa865 (old id 4622255)
- date added to LUP
- 2016-04-01 10:57:56
- date last changed
- 2022-04-12 19:20:01
@article{4c17cb4c-ff2d-48ef-b2f5-c0930c6aa865, abstract = {{In exploratory high-dimensional data analysis, local intrinsic dimension estimation can sometimes be used in order to discriminate between data sets sampled from different low-dimensional structures. Global intrinsic dimension estimators can in many cases be adapted to local estimation, but this leads to problems with high negative bias or high variance. We introduce a method that exploits the curse/blessing of dimensionality and produces local intrinsic dimension estimators that have very low bias, even in cases where the intrinsic dimension is higher than the number of data points, in combination with relatively low variance. We show that our estimators have a very good ability to classify local data sets by their dimension compared to other local intrinsic dimension estimators; furthermore we provide examples showing the usefulness of local intrinsic dimension estimation in general and our method in particular for stratification of real data sets.}}, author = {{Johnsson, Kerstin and Soneson, Charlotte and Fontes, Magnus}}, issn = {{1939-3539}}, keywords = {{Eigenvalues and eigenfunctions; Calibration; Estimation; Dimension; Manifolds}}, language = {{eng}}, number = {{1}}, pages = {{196--202}}, publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}}, series = {{IEEE Transactions on Pattern Analysis and Machine Intelligence}}, title = {{Low Bias Local Intrinsic Dimension Estimation from Expected Simplex Skewness}}, url = {{http://dx.doi.org/10.1109/TPAMI.2014.2343220}}, doi = {{10.1109/TPAMI.2014.2343220}}, volume = {{37}}, year = {{2015}}, }