Bootstrapped Representation Learning for Skeleton-Based Action Recognition

Moliner, Olivier; Huang, Sangxia; Astrom, Kalle (2022). Bootstrapped Representation Learning for Skeleton-Based Action Recognition Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2022, 2022-June,, 4153 - 4163. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2022. New Orleans, United States: IEEE Computer Society
Download:
DOI:
Conference Proceeding/Paper | Published | English
Authors:
Moliner, Olivier ; Huang, Sangxia ; Astrom, Kalle
Department:
Mathematics (Faculty of Engineering)
LTH Profile Area: AI and Digitalization
LTH Profile Area: Engineering Health
eSSENCE: The e-Science Collaboration
Abstract:

In this work, we study self-supervised representation learning for 3D skeleton-based action recognition. We extend Bootstrap Your Own Latent (BYOL) for representation learning on skeleton sequence data and propose a new data augmentation strategy including two asymmetric transformation pipelines. We also introduce a multi-viewpoint sampling method that leverages multiple viewing angles of the same action captured by different cameras. In the semi-supervised setting, we show that the performance can be further improved by knowledge distillation from wider networks, leveraging once more the unlabeled samples. We conduct extensive experiments on the NTU-60, NTU-120 and PKU-MMD datasets to demonstrate the performance of our proposed method. Our method consistently outperforms the current state of the art on linear evaluation, semi-supervised and transfer learning benchmarks.

ISBN:
9781665487399
ISSN:
2160-7508
LUP-ID:
8f121d1e-204c-4910-a1b0-75f138add4d4 | Link: https://lup.lub.lu.se/record/8f121d1e-204c-4910-a1b0-75f138add4d4 | Statistics

Cite this