Advanced

Face Verification and Open-Set Identification for Real-Time Video Applications

Grundström, Jakob LU (2015) In Master's Theses in Mathematical Sciences FMA820 20142
Mathematics (Faculty of Technology) and Numerical Analysis
Abstract (Swedish)
In this work we aim to develop a video-based face verification algorithm suitable for real-time use in an embedded environment with limited space and restricted computational resources. We investigate face verification based solutions to envisioned real-time video applications that require open-set face identification. In particular, we create a prototype system for keeping track of the identities of persons currently inside a closed area. This is done by verifying face images captured at the entry with images captured at the exit.

We consider a pair-wise face verification setup that makes the binary decision if two face images are of matching or non-matching identities. The Joint Bayesian classifier was found useful for this purpose... (More)
In this work we aim to develop a video-based face verification algorithm suitable for real-time use in an embedded environment with limited space and restricted computational resources. We investigate face verification based solutions to envisioned real-time video applications that require open-set face identification. In particular, we create a prototype system for keeping track of the identities of persons currently inside a closed area. This is done by verifying face images captured at the entry with images captured at the exit.

We consider a pair-wise face verification setup that makes the binary decision if two face images are of matching or non-matching identities. The Joint Bayesian classifier was found useful for this purpose and we show that it compares well to a set of standard classifiers. Face verification is evaluated using two distinct feature types: local feature representations around landmark points and deep representations extracted from Convolutional Neural Networks (convnets) trained for generic object detection and fine-tuned on face image data. With these face verification algorithms we implement and evaluate open-set identification.

Combined with a Joint Bayesian classifier the deep representations show good accuracy applied to face verification, we reached a face verification accuracy of 91.37% on the popular Labeled Faces in the Wild benchmark. The feature extraction with the deep representations is computationally demanding but may become applicable to embedded environments in the future. The local feature representations investigated requires less computational resources but yields lower accuracy, 86.4% on the Labeled Faces in the Wild benchmark.

We identified the local features as the currently most applicable feature representation for the considered real-time video applications in the immediate future. For this reason a local feature scheme based on Local Binary Patterns (LBP) is used in our prototype system. For our demo prototype we set up a video-based face recognition pipeline where tracks of face images are matched, instead of matching singular images. Currently, the demo prototype, running on a desktop computer, process two simultaneous video streams either from recorded files or live from network video and maintain a dynamic gallery of the persons that are inside the closed area. (Less)
Please use this url to cite or link to this publication:
author
Grundström, Jakob LU
supervisor
organization
course
FMA820 20142
year
type
H2 - Master's Degree (Two Years)
subject
keywords
computer vision, face verification, open-set identification, real-time video analysis, joint bayesian classifier, local binary pattern, convolutional neural network, fine-tuning, deep representations
publication/series
Master's Theses in Mathematical Sciences
report number
LUTFMA-3273-2015
ISSN
1404-6342
other publication id
2015:E13
language
English
id
5465149
date added to LUP
2015-06-18 12:04:55
date last changed
2015-06-18 12:04:55
@misc{5465149,
  abstract     = {In this work we aim to develop a video-based face verification algorithm suitable for real-time use in an embedded environment with limited space and restricted computational resources. We investigate face verification based solutions to envisioned real-time video applications that require open-set face identification. In particular, we create a prototype system for keeping track of the identities of persons currently inside a closed area. This is done by verifying face images captured at the entry with images captured at the exit.

We consider a pair-wise face verification setup that makes the binary decision if two face images are of matching or non-matching identities. The Joint Bayesian classifier was found useful for this purpose and we show that it compares well to a set of standard classifiers. Face verification is evaluated using two distinct feature types: local feature representations around landmark points and deep representations extracted from Convolutional Neural Networks (convnets) trained for generic object detection and fine-tuned on face image data. With these face verification algorithms we implement and evaluate open-set identification. 

Combined with a Joint Bayesian classifier the deep representations show good accuracy applied to face verification, we reached a face verification accuracy of 91.37% on the popular Labeled Faces in the Wild benchmark. The feature extraction with the deep representations is computationally demanding but may become applicable to embedded environments in the future. The local feature representations investigated requires less computational resources but yields lower accuracy, 86.4% on the Labeled Faces in the Wild benchmark.

We identified the local features as the currently most applicable feature representation for the considered real-time video applications in the immediate future. For this reason a local feature scheme based on Local Binary Patterns (LBP) is used in our prototype system. For our demo prototype we set up a video-based face recognition pipeline where tracks of face images are matched, instead of matching singular images. Currently, the demo prototype, running on a desktop computer, process two simultaneous video streams either from recorded files or live from network video and maintain a dynamic gallery of the persons that are inside the closed area.},
  author       = {Grundström, Jakob},
  issn         = {1404-6342},
  keyword      = {computer vision,face verification,open-set identification,real-time video analysis,joint bayesian classifier,local binary pattern,convolutional neural network,fine-tuning,deep representations},
  language     = {eng},
  note         = {Student Paper},
  series       = {Master's Theses in Mathematical Sciences},
  title        = {Face Verification and Open-Set Identification for Real-Time Video Applications},
  year         = {2015},
}