Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

An FPGA-based On-device Fine-tuning Accelerator for Edge CNN Adaptation

Fan, Yunrui LU and Tang, Mengjing (2026) EITM02 20261
Department of Electrical and Information Technology
Abstract (Swedish)
With the widespread deployment of convolutional neural networks (CNNs) at the
edge, on-device fine-tuning has become essential for real-time model adaptation
to address environmental accuracy degradation. However, on-device training is
constrained by the "memory wall" energy bottleneck and numerical instability from
low-precision fixed-point arithmetic. To tackle these challenges, this thesis presents
an energy-efficient, FPGA-based on-device fine-tuning accelerator featuring a deep
hardware-algorithm co-design.

At the algorithmic level, a partial fine-tuning strategy is adopted to freeze shallow
feature extractors while only updating deeper task-specific layers, reducing the
activation-storage overhead of full backpropagation.... (More)
With the widespread deployment of convolutional neural networks (CNNs) at the
edge, on-device fine-tuning has become essential for real-time model adaptation
to address environmental accuracy degradation. However, on-device training is
constrained by the "memory wall" energy bottleneck and numerical instability from
low-precision fixed-point arithmetic. To tackle these challenges, this thesis presents
an energy-efficient, FPGA-based on-device fine-tuning accelerator featuring a deep
hardware-algorithm co-design.

At the algorithmic level, a partial fine-tuning strategy is adopted to freeze shallow
feature extractors while only updating deeper task-specific layers, reducing the
activation-storage overhead of full backpropagation. Additionally, a hardware-
aware quantization-aware training (HA-QAT) pipeline incorporates LFSR-based
Stochastic Rounding and Stochastic Approximate Pruning (SAP), achieving over
70% backward gradient sparsity with negligible accuracy loss. At the architec-
tural level, forward and backward computations are modeled as a unified seven-
dimensional nested loop mapped onto a custom 2D Processing Element (PE) array.
By employing a Row-Stationary (RS) dataflow and adaptive spatial unrolling, the
design maximizes data reuse and minimizes memory access.

The system is prototyped and verified on the Xilinx Zynq UltraScale+ ZCU104
platform. Evaluations using a lightweight VGG-style network on a perturbed
CIFAR-10 dataset show that the accelerator recovers model accuracy from a de-
graded 69.02% to 84.85% with stochastic pruning enabled (a mere 0.30% drop
versus the unpruned baseline). Operating at 166.67 MHz, it delivers an effective
throughput of 21.93 GOPS with a power consumption of only 2.085 W, yielding
an energy efficiency of 10.52 GOPS/W. This work demonstrates the feasibility of
practical CNN on-device fine-tuning on resource-constrained edge FPGAs, provid-
ing an efficient hardware-software co-design solution for local continuous learning
in edge intelligence. (Less)
Please use this url to cite or link to this publication:
author
Fan, Yunrui LU and Tang, Mengjing
supervisor
organization
course
EITM02 20261
year
type
H2 - Master's Degree (Two Years)
subject
keywords
Edge Computing, On-device Fine-tuning, FPGA, Convolutional Neu- ral Network (CNN), Hardware-Algorithm Co-design, Dataflow Optimization
report number
LU/LTH-EIT 2026-1140
language
English
id
9238144
date added to LUP
2026-06-15 12:51:08
date last changed
2026-06-15 12:51:08
@misc{9238144,
  abstract     = {{With the widespread deployment of convolutional neural networks (CNNs) at the
edge, on-device fine-tuning has become essential for real-time model adaptation
to address environmental accuracy degradation. However, on-device training is
constrained by the "memory wall" energy bottleneck and numerical instability from
low-precision fixed-point arithmetic. To tackle these challenges, this thesis presents
an energy-efficient, FPGA-based on-device fine-tuning accelerator featuring a deep
hardware-algorithm co-design.

At the algorithmic level, a partial fine-tuning strategy is adopted to freeze shallow
feature extractors while only updating deeper task-specific layers, reducing the
activation-storage overhead of full backpropagation. Additionally, a hardware-
aware quantization-aware training (HA-QAT) pipeline incorporates LFSR-based
Stochastic Rounding and Stochastic Approximate Pruning (SAP), achieving over
70% backward gradient sparsity with negligible accuracy loss. At the architec-
tural level, forward and backward computations are modeled as a unified seven-
dimensional nested loop mapped onto a custom 2D Processing Element (PE) array.
By employing a Row-Stationary (RS) dataflow and adaptive spatial unrolling, the
design maximizes data reuse and minimizes memory access.

The system is prototyped and verified on the Xilinx Zynq UltraScale+ ZCU104
platform. Evaluations using a lightweight VGG-style network on a perturbed
CIFAR-10 dataset show that the accelerator recovers model accuracy from a de-
graded 69.02% to 84.85% with stochastic pruning enabled (a mere 0.30% drop
versus the unpruned baseline). Operating at 166.67 MHz, it delivers an effective
throughput of 21.93 GOPS with a power consumption of only 2.085 W, yielding
an energy efficiency of 10.52 GOPS/W. This work demonstrates the feasibility of
practical CNN on-device fine-tuning on resource-constrained edge FPGAs, provid-
ing an efficient hardware-software co-design solution for local continuous learning
in edge intelligence.}},
  author       = {{Fan, Yunrui and Tang, Mengjing}},
  language     = {{eng}},
  note         = {{Student Paper}},
  title        = {{An FPGA-based On-device Fine-tuning Accelerator for Edge CNN Adaptation}},
  year         = {{2026}},
}