Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

A Real-Time Gesture Recognition System with FPGA Accelerated ZynqNet Classification

Nunez-Prieto, Ricardo ; Gomez, Pablo Correa and Liu, Liang LU orcid (2019) 5th IEEE Nordic Circuits and Systems Conference, NORCAS 2019: NORCHIP and International Symposium of System-on-Chip, SoC 2019
Abstract

This paper presents a real-time hand gesture recognition system by accelerating a convolutional neural network (CNN) using FPGA platform. More specifically, ZynqNet is adopted and modified to fulfill the classification task of recognizing the Swedish manual alphabet, which is used by sign language users for spelling purposes, also known as fingerspelling. Data augmentation and transfer learning techniques have been used during the training phase to improve the classification accuracy up to 80.1%, even with an 8-bit ZynqNet model. Extensive analysis of memory requirements and data processing patterns has been performed to enable optimization techniques, including memory partitioning and register arrays. The resulting FPGA implementation... (More)

This paper presents a real-time hand gesture recognition system by accelerating a convolutional neural network (CNN) using FPGA platform. More specifically, ZynqNet is adopted and modified to fulfill the classification task of recognizing the Swedish manual alphabet, which is used by sign language users for spelling purposes, also known as fingerspelling. Data augmentation and transfer learning techniques have been used during the training phase to improve the classification accuracy up to 80.1%, even with an 8-bit ZynqNet model. Extensive analysis of memory requirements and data processing patterns has been performed to enable optimization techniques, including memory partitioning and register arrays. The resulting FPGA implementation on a Xilinx UltraScale device avoids the use of off-chip memories, which together with block-wise processing scheduling, achieves an image rate of 23.5 frames per second (FPS) at 200 MHz clock frequency.

(Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
CNN, computer vision, FPGA, hand gesture recognition, high-level synthesis, ZynqNet
host publication
2019 IEEE Nordic Circuits and Systems Conference, NORCAS 2019 : NORCHIP and International Symposium of System-on-Chip, SoC 2019 - Proceedings - NORCHIP and International Symposium of System-on-Chip, SoC 2019 - Proceedings
editor
Nurmi, Jari ; Ellervee, Peeter ; Halonen, Kari and Roning, Juha
article number
8906956
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
conference name
5th IEEE Nordic Circuits and Systems Conference, NORCAS 2019: NORCHIP and International Symposium of System-on-Chip, SoC 2019
conference location
Helsinki, Finland
conference dates
2019-10-29 - 2019-10-30
external identifiers
  • scopus:85076009713
ISBN
9781728127699
978-1-7281-2770-5
DOI
10.1109/NORCHIP.2019.8906956
language
English
LU publication?
yes
id
9f0cc249-7a63-4a77-8410-9b1a2264e53c
date added to LUP
2020-01-03 11:25:48
date last changed
2024-05-15 04:12:34
@inproceedings{9f0cc249-7a63-4a77-8410-9b1a2264e53c,
  abstract     = {{<p>This paper presents a real-time hand gesture recognition system by accelerating a convolutional neural network (CNN) using FPGA platform. More specifically, ZynqNet is adopted and modified to fulfill the classification task of recognizing the Swedish manual alphabet, which is used by sign language users for spelling purposes, also known as fingerspelling. Data augmentation and transfer learning techniques have been used during the training phase to improve the classification accuracy up to 80.1%, even with an 8-bit ZynqNet model. Extensive analysis of memory requirements and data processing patterns has been performed to enable optimization techniques, including memory partitioning and register arrays. The resulting FPGA implementation on a Xilinx UltraScale device avoids the use of off-chip memories, which together with block-wise processing scheduling, achieves an image rate of 23.5 frames per second (FPS) at 200 MHz clock frequency.</p>}},
  author       = {{Nunez-Prieto, Ricardo and Gomez, Pablo Correa and Liu, Liang}},
  booktitle    = {{2019 IEEE Nordic Circuits and Systems Conference, NORCAS 2019 : NORCHIP and International Symposium of System-on-Chip, SoC 2019 - Proceedings}},
  editor       = {{Nurmi, Jari and Ellervee, Peeter and Halonen, Kari and Roning, Juha}},
  isbn         = {{9781728127699}},
  keywords     = {{CNN; computer vision; FPGA; hand gesture recognition; high-level synthesis; ZynqNet}},
  language     = {{eng}},
  month        = {{11}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  title        = {{A Real-Time Gesture Recognition System with FPGA Accelerated ZynqNet Classification}},
  url          = {{http://dx.doi.org/10.1109/NORCHIP.2019.8906956}},
  doi          = {{10.1109/NORCHIP.2019.8906956}},
  year         = {{2019}},
}