Alternative representations of the differentiable spectrogram as a trainable layer in a neural network
(2026) In Master’s Theses in Mathematical Sciences FMSM01 20261Mathematical Statistics
- Abstract
- The multitaper spectrogram, with arithmetic and geometric mean functions, and the ambiguity domain where evaluated on simulated data and real audio samples{. This was done} to determine if they can improve the results, of classification tasks, of the single-taper spectrogram when used as input to a convolutional neural network. The spectrogram used was differentiable to allow for the window length to be learnable during the training process. Results show that the multitaper spectrogram can perform better for noisy data, with combining the tapers with the geometric mean instead of the usual arithmetic mean yielding better results. The correct number of tapers were required to be selected as it would stop performing better otherwise. There... (More)
- The multitaper spectrogram, with arithmetic and geometric mean functions, and the ambiguity domain where evaluated on simulated data and real audio samples{. This was done} to determine if they can improve the results, of classification tasks, of the single-taper spectrogram when used as input to a convolutional neural network. The spectrogram used was differentiable to allow for the window length to be learnable during the training process. Results show that the multitaper spectrogram can perform better for noisy data, with combining the tapers with the geometric mean instead of the usual arithmetic mean yielding better results. The correct number of tapers were required to be selected as it would stop performing better otherwise. There is no clear way to select a good number of tapers beforehand and without trial and error. Furthermore it does not show much promise in the real audio samples as it did not yield an improvement over the single-taper spectrogram and comes with increased memory and computational requirements.
The ambiguity domain performed the best when it was represented with the absolute value as input to the neural network. However, it performs worse than the spectrogram and also comes with increased computational requirement. This causes it to be undesirable to use for general audio classification tasks.
Both the ambiguity function and multitaper spectrogram does seem more resilient to the choice of window length however, as it had a larger standard deviation than the single-taper spectrogram while having comparable results. Additionally, having trainable weights for the mean functions can help the arithmetic mean perform better than having static and equal weights for all tapers. This does not seem to be the case for the geometric mean function as having equal weights does not perform worse. As a whole the single-taper spectrogram still seems to be the better choice than these alternative representations and comes with lower memory and computational complexity. (Less) - Popular Abstract
- Spectrograms are an important tool for representing signals, such as audio, as images. These can then be analyzed by a neural network to guess what the signal represents. In this thesis alternative representations of the spectrograms were studied as inputs.
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/student-papers/record/9243782
- author
- Vadebo, Elias LU and Vågsäter, Yui LU
- supervisor
- organization
- alternative title
- Alternativa representationer av det differentierbara spektrogrammet som ett träningsbart lager i ett neuralt nätverk
- course
- FMSM01 20261
- year
- 2026
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- Audio signal, classification, convolutional neural network (CNN), discrete-time signal, learnable ambiguity function, learnable spectrogram, multitaper spectrogram
- publication/series
- Master’s Theses in Mathematical Sciences
- report number
- LUTFMS-3568-2026
- ISSN
- 1404-6342
- other publication id
- 2026:E94
- language
- English
- id
- 9243782
- date added to LUP
- 2026-06-25 09:25:40
- date last changed
- 2026-06-25 09:25:40
@misc{9243782,
abstract = {{The multitaper spectrogram, with arithmetic and geometric mean functions, and the ambiguity domain where evaluated on simulated data and real audio samples{. This was done} to determine if they can improve the results, of classification tasks, of the single-taper spectrogram when used as input to a convolutional neural network. The spectrogram used was differentiable to allow for the window length to be learnable during the training process. Results show that the multitaper spectrogram can perform better for noisy data, with combining the tapers with the geometric mean instead of the usual arithmetic mean yielding better results. The correct number of tapers were required to be selected as it would stop performing better otherwise. There is no clear way to select a good number of tapers beforehand and without trial and error. Furthermore it does not show much promise in the real audio samples as it did not yield an improvement over the single-taper spectrogram and comes with increased memory and computational requirements.
The ambiguity domain performed the best when it was represented with the absolute value as input to the neural network. However, it performs worse than the spectrogram and also comes with increased computational requirement. This causes it to be undesirable to use for general audio classification tasks.
Both the ambiguity function and multitaper spectrogram does seem more resilient to the choice of window length however, as it had a larger standard deviation than the single-taper spectrogram while having comparable results. Additionally, having trainable weights for the mean functions can help the arithmetic mean perform better than having static and equal weights for all tapers. This does not seem to be the case for the geometric mean function as having equal weights does not perform worse. As a whole the single-taper spectrogram still seems to be the better choice than these alternative representations and comes with lower memory and computational complexity.}},
author = {{Vadebo, Elias and Vågsäter, Yui}},
issn = {{1404-6342}},
language = {{eng}},
note = {{Student Paper}},
series = {{Master’s Theses in Mathematical Sciences}},
title = {{Alternative representations of the differentiable spectrogram as a trainable layer in a neural network}},
year = {{2026}},
}