Face Verification and Open-Set Identification for Real-Time Video Applications
(2015) In Master's Theses in Mathematical Sciences FMA820 20142Mathematics (Faculty of Engineering)
- Abstract (Swedish)
- In this work we aim to develop a video-based face verification algorithm suitable for real-time use in an embedded environment with limited space and restricted computational resources. We investigate face verification based solutions to envisioned real-time video applications that require open-set face identification. In particular, we create a prototype system for keeping track of the identities of persons currently inside a closed area. This is done by verifying face images captured at the entry with images captured at the exit.
We consider a pair-wise face verification setup that makes the binary decision if two face images are of matching or non-matching identities. The Joint Bayesian classifier was found useful for this purpose... (More) - In this work we aim to develop a video-based face verification algorithm suitable for real-time use in an embedded environment with limited space and restricted computational resources. We investigate face verification based solutions to envisioned real-time video applications that require open-set face identification. In particular, we create a prototype system for keeping track of the identities of persons currently inside a closed area. This is done by verifying face images captured at the entry with images captured at the exit.
We consider a pair-wise face verification setup that makes the binary decision if two face images are of matching or non-matching identities. The Joint Bayesian classifier was found useful for this purpose and we show that it compares well to a set of standard classifiers. Face verification is evaluated using two distinct feature types: local feature representations around landmark points and deep representations extracted from Convolutional Neural Networks (convnets) trained for generic object detection and fine-tuned on face image data. With these face verification algorithms we implement and evaluate open-set identification.
Combined with a Joint Bayesian classifier the deep representations show good accuracy applied to face verification, we reached a face verification accuracy of 91.37% on the popular Labeled Faces in the Wild benchmark. The feature extraction with the deep representations is computationally demanding but may become applicable to embedded environments in the future. The local feature representations investigated requires less computational resources but yields lower accuracy, 86.4% on the Labeled Faces in the Wild benchmark.
We identified the local features as the currently most applicable feature representation for the considered real-time video applications in the immediate future. For this reason a local feature scheme based on Local Binary Patterns (LBP) is used in our prototype system. For our demo prototype we set up a video-based face recognition pipeline where tracks of face images are matched, instead of matching singular images. Currently, the demo prototype, running on a desktop computer, process two simultaneous video streams either from recorded files or live from network video and maintain a dynamic gallery of the persons that are inside the closed area. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/5465149
- author
- Grundström, Jakob LU
- supervisor
- organization
- course
- FMA820 20142
- year
- 2015
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- computer vision, face verification, open-set identification, real-time video analysis, joint bayesian classifier, local binary pattern, convolutional neural network, fine-tuning, deep representations
- publication/series
- Master's Theses in Mathematical Sciences
- report number
- LUTFMA-3273-2015
- ISSN
- 1404-6342
- other publication id
- 2015:E13
- language
- English
- id
- 5465149
- date added to LUP
- 2015-06-18 12:04:55
- date last changed
- 2015-06-18 12:04:55
@misc{5465149,
abstract = {{In this work we aim to develop a video-based face verification algorithm suitable for real-time use in an embedded environment with limited space and restricted computational resources. We investigate face verification based solutions to envisioned real-time video applications that require open-set face identification. In particular, we create a prototype system for keeping track of the identities of persons currently inside a closed area. This is done by verifying face images captured at the entry with images captured at the exit.
We consider a pair-wise face verification setup that makes the binary decision if two face images are of matching or non-matching identities. The Joint Bayesian classifier was found useful for this purpose and we show that it compares well to a set of standard classifiers. Face verification is evaluated using two distinct feature types: local feature representations around landmark points and deep representations extracted from Convolutional Neural Networks (convnets) trained for generic object detection and fine-tuned on face image data. With these face verification algorithms we implement and evaluate open-set identification.
Combined with a Joint Bayesian classifier the deep representations show good accuracy applied to face verification, we reached a face verification accuracy of 91.37% on the popular Labeled Faces in the Wild benchmark. The feature extraction with the deep representations is computationally demanding but may become applicable to embedded environments in the future. The local feature representations investigated requires less computational resources but yields lower accuracy, 86.4% on the Labeled Faces in the Wild benchmark.
We identified the local features as the currently most applicable feature representation for the considered real-time video applications in the immediate future. For this reason a local feature scheme based on Local Binary Patterns (LBP) is used in our prototype system. For our demo prototype we set up a video-based face recognition pipeline where tracks of face images are matched, instead of matching singular images. Currently, the demo prototype, running on a desktop computer, process two simultaneous video streams either from recorded files or live from network video and maintain a dynamic gallery of the persons that are inside the closed area.}},
author = {{Grundström, Jakob}},
issn = {{1404-6342}},
language = {{eng}},
note = {{Student Paper}},
series = {{Master's Theses in Mathematical Sciences}},
title = {{Face Verification and Open-Set Identification for Real-Time Video Applications}},
year = {{2015}},
}