Binary Forward-Only Algorithms
Huang, Baichuan LU
and Aminifar, Amir
LU
(2025)
In IEEE Design and Test
- Abstract
- Today, the overwhelming majority of Internet of Things (IoT) and mobile edge devices have extreme resource limitations, e.g., in terms of computing, memory, and energy. As a result, training Deep Neural Networks (DNNs) using the complex Backpropagation (BP) algorithm on such edge devices presents a major challenge. Forward-only algorithms have emerged as more computation- and memory-efficient alternatives without the requirement for backward passes. In this paper, we investigate binarizing state-of-the-art forward-only algorithms, which are applied to the forward passes of PEPITA, FF, and CwComp. We evaluate these forward-only algorithms with binarization and demonstrate that weight-only binarization may be up to ~31× more efficient in... (More)
- Today, the overwhelming majority of Internet of Things (IoT) and mobile edge devices have extreme resource limitations, e.g., in terms of computing, memory, and energy. As a result, training Deep Neural Networks (DNNs) using the complex Backpropagation (BP) algorithm on such edge devices presents a major challenge. Forward-only algorithms have emerged as more computation- and memory-efficient alternatives without the requirement for backward passes. In this paper, we investigate binarizing state-of-the-art forward-only algorithms, which are applied to the forward passes of PEPITA, FF, and CwComp. We evaluate these forward-only algorithms with binarization and demonstrate that weight-only binarization may be up to ~31× more efficient in terms of memory, with minor degradation in classification performance. Furthermore, we investigate and compare BP and forward-only algorithms in terms of binarization, finding that PEPITA and FF are more vulnerable to binary activations. The code is available at https://github.com/whubaichuan/BinaryFO. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/5015f559-9a0e-471d-a178-dadea762c97a
- author
- Huang, Baichuan
LU
and Aminifar, Amir
LU
- organization
-
- LTH Profile Area: Engineering Health
- Secure and Networked Systems
- ELLIIT: the Linköping-Lund initiative on IT and mobile communication
- LTH Profile Area: AI and Digitalization
- LU Profile Area: Natural and Artificial Cognition
- LTH Profile Area: Water
- NEXTG2COM – a Vinnova Competence Centre in Advanced Digitalisation
- publishing date
- 2025-01-01
- type
- Contribution to journal
- publication status
- epub
- subject
- in
- IEEE Design and Test
- pages
- 6 pages
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- external identifiers
-
- scopus:85214798656
- ISSN
- 2168-2356
- DOI
- 10.1109/MDAT.2025.3528366
- language
- English
- LU publication?
- yes
- id
- 5015f559-9a0e-471d-a178-dadea762c97a
- date added to LUP
- 2025-03-19 15:40:29
- date last changed
- 2025-05-29 09:17:57
@article{5015f559-9a0e-471d-a178-dadea762c97a,
abstract = {{Today, the overwhelming majority of Internet of Things (IoT) and mobile edge devices have extreme resource limitations, e.g., in terms of computing, memory, and energy. As a result, training Deep Neural Networks (DNNs) using the complex Backpropagation (BP) algorithm on such edge devices presents a major challenge. Forward-only algorithms have emerged as more computation- and memory-efficient alternatives without the requirement for backward passes. In this paper, we investigate binarizing state-of-the-art forward-only algorithms, which are applied to the forward passes of PEPITA, FF, and CwComp. We evaluate these forward-only algorithms with binarization and demonstrate that weight-only binarization may be up to ~31× more efficient in terms of memory, with minor degradation in classification performance. Furthermore, we investigate and compare BP and forward-only algorithms in terms of binarization, finding that PEPITA and FF are more vulnerable to binary activations. The code is available at https://github.com/whubaichuan/BinaryFO.}},
author = {{Huang, Baichuan and Aminifar, Amir}},
issn = {{2168-2356}},
language = {{eng}},
month = {{01}},
publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
series = {{IEEE Design and Test}},
title = {{Binary Forward-Only Algorithms}},
url = {{http://dx.doi.org/10.1109/MDAT.2025.3528366}},
doi = {{10.1109/MDAT.2025.3528366}},
year = {{2025}},
}