A Real-Time Gesture Recognition System with FPGA Accelerated ZynqNet Classification
(2019) 5th IEEE Nordic Circuits and Systems Conference, NORCAS 2019: NORCHIP and International Symposium of System-on-Chip, SoC 2019- Abstract
This paper presents a real-time hand gesture recognition system by accelerating a convolutional neural network (CNN) using FPGA platform. More specifically, ZynqNet is adopted and modified to fulfill the classification task of recognizing the Swedish manual alphabet, which is used by sign language users for spelling purposes, also known as fingerspelling. Data augmentation and transfer learning techniques have been used during the training phase to improve the classification accuracy up to 80.1%, even with an 8-bit ZynqNet model. Extensive analysis of memory requirements and data processing patterns has been performed to enable optimization techniques, including memory partitioning and register arrays. The resulting FPGA implementation... (More)
This paper presents a real-time hand gesture recognition system by accelerating a convolutional neural network (CNN) using FPGA platform. More specifically, ZynqNet is adopted and modified to fulfill the classification task of recognizing the Swedish manual alphabet, which is used by sign language users for spelling purposes, also known as fingerspelling. Data augmentation and transfer learning techniques have been used during the training phase to improve the classification accuracy up to 80.1%, even with an 8-bit ZynqNet model. Extensive analysis of memory requirements and data processing patterns has been performed to enable optimization techniques, including memory partitioning and register arrays. The resulting FPGA implementation on a Xilinx UltraScale device avoids the use of off-chip memories, which together with block-wise processing scheduling, achieves an image rate of 23.5 frames per second (FPS) at 200 MHz clock frequency.
(Less)
- author
- Nunez-Prieto, Ricardo ; Gomez, Pablo Correa and Liu, Liang LU
- organization
- publishing date
- 2019-11-21
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- keywords
- CNN, computer vision, FPGA, hand gesture recognition, high-level synthesis, ZynqNet
- host publication
- 2019 IEEE Nordic Circuits and Systems Conference, NORCAS 2019 : NORCHIP and International Symposium of System-on-Chip, SoC 2019 - Proceedings - NORCHIP and International Symposium of System-on-Chip, SoC 2019 - Proceedings
- editor
- Nurmi, Jari ; Ellervee, Peeter ; Halonen, Kari and Roning, Juha
- article number
- 8906956
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- conference name
- 5th IEEE Nordic Circuits and Systems Conference, NORCAS 2019: NORCHIP and International Symposium of System-on-Chip, SoC 2019
- conference location
- Helsinki, Finland
- conference dates
- 2019-10-29 - 2019-10-30
- external identifiers
-
- scopus:85076009713
- ISBN
- 978-1-7281-2770-5
- 9781728127699
- DOI
- 10.1109/NORCHIP.2019.8906956
- language
- English
- LU publication?
- yes
- id
- 9f0cc249-7a63-4a77-8410-9b1a2264e53c
- date added to LUP
- 2020-01-03 11:25:48
- date last changed
- 2024-08-21 13:39:43
@inproceedings{9f0cc249-7a63-4a77-8410-9b1a2264e53c, abstract = {{<p>This paper presents a real-time hand gesture recognition system by accelerating a convolutional neural network (CNN) using FPGA platform. More specifically, ZynqNet is adopted and modified to fulfill the classification task of recognizing the Swedish manual alphabet, which is used by sign language users for spelling purposes, also known as fingerspelling. Data augmentation and transfer learning techniques have been used during the training phase to improve the classification accuracy up to 80.1%, even with an 8-bit ZynqNet model. Extensive analysis of memory requirements and data processing patterns has been performed to enable optimization techniques, including memory partitioning and register arrays. The resulting FPGA implementation on a Xilinx UltraScale device avoids the use of off-chip memories, which together with block-wise processing scheduling, achieves an image rate of 23.5 frames per second (FPS) at 200 MHz clock frequency.</p>}}, author = {{Nunez-Prieto, Ricardo and Gomez, Pablo Correa and Liu, Liang}}, booktitle = {{2019 IEEE Nordic Circuits and Systems Conference, NORCAS 2019 : NORCHIP and International Symposium of System-on-Chip, SoC 2019 - Proceedings}}, editor = {{Nurmi, Jari and Ellervee, Peeter and Halonen, Kari and Roning, Juha}}, isbn = {{978-1-7281-2770-5}}, keywords = {{CNN; computer vision; FPGA; hand gesture recognition; high-level synthesis; ZynqNet}}, language = {{eng}}, month = {{11}}, publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}}, title = {{A Real-Time Gesture Recognition System with FPGA Accelerated ZynqNet Classification}}, url = {{http://dx.doi.org/10.1109/NORCHIP.2019.8906956}}, doi = {{10.1109/NORCHIP.2019.8906956}}, year = {{2019}}, }