Advantages of binary stochastic synapses for hardware spiking neural networks with realistic memristors
Sulinskas, Karolis and Borg, Mattias LU
(2022)
In Neuromorphic Computing and Engineering
2(3).
- Abstract
Hardware implementing spiking neural networks (SNNs) has the potential to provide transformative gains in energy efficiency and throughput for energy-restricted machine-learning tasks. This is enabled by large arrays of memristive synapse devices that can be realized by various emerging memory technologies. But in practice, the performance of such hardware is limited by non-ideal features of the memristor devices such as nonlinear and asymmetric state updates, limited bit-resolution, limited cycling endurance and device noise. Here we investigate how stochastic switching in binary synapses can provide advantages compared with realistic analog memristors when using unsupervised training of SNNs via spike timing-dependent plasticity. We... (More)
Hardware implementing spiking neural networks (SNNs) has the potential to provide transformative gains in energy efficiency and throughput for energy-restricted machine-learning tasks. This is enabled by large arrays of memristive synapse devices that can be realized by various emerging memory technologies. But in practice, the performance of such hardware is limited by non-ideal features of the memristor devices such as nonlinear and asymmetric state updates, limited bit-resolution, limited cycling endurance and device noise. Here we investigate how stochastic switching in binary synapses can provide advantages compared with realistic analog memristors when using unsupervised training of SNNs via spike timing-dependent plasticity. We find that the performance of binary stochastic SNNs is similar to or even better than analog deterministic SNNs when one considers memristors with realistic bit-resolution as well in situations with considerable cycle-to-cycle noise. Furthermore, binary stochastic SNNs require many fewer weight updates to train, leading to superior utilization of the limited endurance in realistic memristive devices.
(Less)
- author
- Sulinskas, Karolis
and Borg, Mattias
LU
- organization
- publishing date
- 2022-09-01
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- binary, memristor, MNIST, spiking neural networks, stochastic
- in
- Neuromorphic Computing and Engineering
- volume
- 2
- issue
- 3
- article number
- 034008
- publisher
- IOP Publishing
- external identifiers
-
- scopus:85149318326
- ISSN
- 2634-4386
- DOI
- 10.1088/2634-4386/ac7c89
- project
- Robust neuromorphic computing using ferroelectric memristors
- language
- English
- LU publication?
- yes
- additional info
- Funding Information: This work was supported by the Swedish Foundation for Strategic Research (SSF) project no. SM21-0008, and the Swedish Research Counsel (VR) project no. 2018-05379. The authors acknowledge the helpful input of Dr Saeed Bastani. Publisher Copyright: © 2022 The Author(s). Published by IOP Publishing Ltd.
- id
- 4fe8a519-dafd-4599-8489-cb948aebf60a
- date added to LUP
- 2023-06-16 10:59:51
- date last changed
- 2023-11-08 07:01:56
@article{4fe8a519-dafd-4599-8489-cb948aebf60a,
abstract = {{<p>Hardware implementing spiking neural networks (SNNs) has the potential to provide transformative gains in energy efficiency and throughput for energy-restricted machine-learning tasks. This is enabled by large arrays of memristive synapse devices that can be realized by various emerging memory technologies. But in practice, the performance of such hardware is limited by non-ideal features of the memristor devices such as nonlinear and asymmetric state updates, limited bit-resolution, limited cycling endurance and device noise. Here we investigate how stochastic switching in binary synapses can provide advantages compared with realistic analog memristors when using unsupervised training of SNNs via spike timing-dependent plasticity. We find that the performance of binary stochastic SNNs is similar to or even better than analog deterministic SNNs when one considers memristors with realistic bit-resolution as well in situations with considerable cycle-to-cycle noise. Furthermore, binary stochastic SNNs require many fewer weight updates to train, leading to superior utilization of the limited endurance in realistic memristive devices.</p>}},
author = {{Sulinskas, Karolis and Borg, Mattias}},
issn = {{2634-4386}},
keywords = {{binary; memristor; MNIST; spiking neural networks; stochastic}},
language = {{eng}},
month = {{09}},
number = {{3}},
publisher = {{IOP Publishing}},
series = {{Neuromorphic Computing and Engineering}},
title = {{Advantages of binary stochastic synapses for hardware spiking neural networks with realistic memristors}},
url = {{http://dx.doi.org/10.1088/2634-4386/ac7c89}},
doi = {{10.1088/2634-4386/ac7c89}},
volume = {{2}},
year = {{2022}},
}