Advanced

Capturing Detailed Hand Motion Using the Kinect Sensor and Max-Sum Belief Propagation

Bore, Nils LU and Holmqvist, Mikael LU (2012) In LUTFMA-3236-2012 FMA820 20122
Mathematics (Faculty of Technology) and Numerical Analysis
Abstract
Recent research indicates that several neurological diseases that affect motor functions could be diagnosed using analysis of detailed arm and hand motion. This analysis has earlier been carried out manually by looking for certain mo- tion patterns in patients and animals performing a skilled reaching task. In this thesis we investigate the possibility of performing these tests in a more auto- mated fashion by implementing image analysis methods for capturing arm and hand motion data from RGBD recordings. We have used the Microsoft Kinect sensor to capture motion both on a precise level, describing movements around individual joints of the hand, and on a coarser level, finding directions and po- sitions of the lower and upper arm.
Our... (More)
Recent research indicates that several neurological diseases that affect motor functions could be diagnosed using analysis of detailed arm and hand motion. This analysis has earlier been carried out manually by looking for certain mo- tion patterns in patients and animals performing a skilled reaching task. In this thesis we investigate the possibility of performing these tests in a more auto- mated fashion by implementing image analysis methods for capturing arm and hand motion data from RGBD recordings. We have used the Microsoft Kinect sensor to capture motion both on a precise level, describing movements around individual joints of the hand, and on a coarser level, finding directions and po- sitions of the lower and upper arm.
Our methods take advantage of both the RGB photos, detecting skin colour and finding arm/hand pixels, and the depth images, constructing 3D point clouds that we try to match to a simple geometrical model of the hand. Our approach is to model each phalanx of the hand individually, draw hypotheses for each of these around their pose from the previous frame and then optimize to find the most likely hand configuration using a Belief Propagation based algorithm.
We present results from running our algorithms on a few test sequences. The algorithm works well under favourable circumstances but has problems giving the correct pose for example when fingers occlude each other. Possible additions to the framework that might help to overcome these issues are also discussed. (Less)
Please use this url to cite or link to this publication:
author
Bore, Nils LU and Holmqvist, Mikael LU
supervisor
organization
alternative title
Följning av detaljerade handrörelser med en Kinect-sensor och Max-Sum Belief Propagation
course
FMA820 20122
year
type
H2 - Master's Degree (Two Years)
subject
keywords
hand tracking, belief propagation, message passing, image analysis, kinect, max-sum algorithm, ransac, skin colour segmentation, detailed hand tracking
publication/series
LUTFMA-3236-2012
report number
LUTFMA-3236-2012
ISSN
1404-6342
other publication id
2012:E42
language
English
id
3232328
date added to LUP
2013-04-25 17:27:32
date last changed
2013-04-25 17:27:32
@misc{3232328,
  abstract     = {Recent research indicates that several neurological diseases that affect motor functions could be diagnosed using analysis of detailed arm and hand motion. This analysis has earlier been carried out manually by looking for certain mo- tion patterns in patients and animals performing a skilled reaching task. In this thesis we investigate the possibility of performing these tests in a more auto- mated fashion by implementing image analysis methods for capturing arm and hand motion data from RGBD recordings. We have used the Microsoft Kinect sensor to capture motion both on a precise level, describing movements around individual joints of the hand, and on a coarser level, finding directions and po- sitions of the lower and upper arm.
Our methods take advantage of both the RGB photos, detecting skin colour and finding arm/hand pixels, and the depth images, constructing 3D point clouds that we try to match to a simple geometrical model of the hand. Our approach is to model each phalanx of the hand individually, draw hypotheses for each of these around their pose from the previous frame and then optimize to find the most likely hand configuration using a Belief Propagation based algorithm.
We present results from running our algorithms on a few test sequences. The algorithm works well under favourable circumstances but has problems giving the correct pose for example when fingers occlude each other. Possible additions to the framework that might help to overcome these issues are also discussed.},
  author       = {Bore, Nils and Holmqvist, Mikael},
  issn         = {1404-6342},
  keyword      = {hand tracking,belief propagation,message passing,image analysis,kinect,max-sum algorithm,ransac,skin colour segmentation,detailed hand tracking},
  language     = {eng},
  note         = {Student Paper},
  series       = {LUTFMA-3236-2012},
  title        = {Capturing Detailed Hand Motion Using the Kinect Sensor and Max-Sum Belief Propagation},
  year         = {2012},
}