Advanced

First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition

Gharaee, Zahra LU ; Gärdenfors, Peter LU and Johnsson, Magnus LU (2017) In Applied Soft Computing 59. p.574-585
Abstract
Human recognition of the actions of other humans is very efficient and is based on patterns of movements. Our theoretical starting point is that the dynamics of the joint movements is important to action categorization. On the basis of this theory, we present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics. We evaluate our system in two experiments with publicly available datasets, and compare its performance to the performance with less sophisticated... (More)
Human recognition of the actions of other humans is very efficient and is based on patterns of movements. Our theoretical starting point is that the dynamics of the joint movements is important to action categorization. On the basis of this theory, we present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics. We evaluate our system in two experiments with publicly available datasets, and compare its performance to the performance with less sophisticated preprocessing of the input. The results show that including the dynamics of the actions improves the performance. We also apply an attention mechanism that focuses on the parts of the body that are the most involved in performing the actions. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
Applied Soft Computing
volume
59
pages
574 - 585
publisher
Elsevier
external identifiers
  • scopus:85021095829
  • wos:000407732600042
ISSN
1568-4946
DOI
10.1016/j.asoc.2017.06.007
project
What you see is what you do (WYSIWYD)
language
English
LU publication?
yes
id
744159df-9a31-420e-9e16-29a10b433723 (old id 8626405)
alternative location
https://authors.elsevier.com/a/1VGhU5aecSVndE
date added to LUP
2016-02-15 08:20:46
date last changed
2018-01-16 13:19:49
@article{744159df-9a31-420e-9e16-29a10b433723,
  abstract     = {Human recognition of the actions of other humans is very efficient and is based on patterns of movements. Our theoretical starting point is that the dynamics of the joint movements is important to action categorization. On the basis of this theory, we present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics. We evaluate our system in two experiments with publicly available datasets, and compare its performance to the performance with less sophisticated preprocessing of the input. The results show that including the dynamics of the actions improves the performance. We also apply an attention mechanism that focuses on the parts of the body that are the most involved in performing the actions.},
  author       = {Gharaee, Zahra and Gärdenfors, Peter and Johnsson, Magnus},
  issn         = {1568-4946},
  language     = {eng},
  month        = {06},
  pages        = {574--585},
  publisher    = {Elsevier},
  series       = {Applied Soft Computing},
  title        = {First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition},
  url          = {http://dx.doi.org/10.1016/j.asoc.2017.06.007},
  volume       = {59},
  year         = {2017},
}