Acoustic Degradation Affects Neural Measures of Speech Tracking and Understanding
(2025)Department of Automatic Control
- Abstract
- This study investigated how acoustical challenges, specifically changes in signal-to-noise ratio (SNR), affect the neural tracking of attended and ignored speech in individuals with hearing impairment. The analysis began with the speech envelope and was progressively extended by incorporating phonetic information as well as outputs from all layers of OpenAI’s Whisper model. The goal was to gain a deeper understanding of the brains’ audio processing pathways during Cocktail Party Problem like situations. Using a Temporal Respons Function to model brain activity, the results showed that adding these features improved the model’s ability to predict neural responses, with the correlation between predicted and recorded EEG increasing from 3.2%... (More)
- This study investigated how acoustical challenges, specifically changes in signal-to-noise ratio (SNR), affect the neural tracking of attended and ignored speech in individuals with hearing impairment. The analysis began with the speech envelope and was progressively extended by incorporating phonetic information as well as outputs from all layers of OpenAI’s Whisper model. The goal was to gain a deeper understanding of the brains’ audio processing pathways during Cocktail Party Problem like situations. Using a Temporal Respons Function to model brain activity, the results showed that adding these features improved the model’s ability to predict neural responses, with the correlation between predicted and recorded EEG increasing from 3.2% when using only the envelope to 3.8% when combining the envelope, phonemes, and Whisper’s layer-wise outputs (p = 0.0021). When distinguishing neural responses between attended and ignored speech, the inclusion of these additional features did not influence the classification accuracy significantly compared to models using only basic acoustic features.
This work has potential applications in hearing technology, enabling the assessment of hearing and the effects of various hearing devices. Furthermore, it could inform signal processing algorithms in hearing aids, enhancing speech comprehension tracking and improving auditory attention by tailoring hearing devices to the specific needs of individual users. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9206066
- author
- Müller, Josefine and Thulin, Cajsa
- supervisor
- organization
- year
- 2025
- type
- H3 - Professional qualifications (4 Years - )
- subject
- report number
- TFRT-6277
- other publication id
- 0280-5316
- language
- English
- id
- 9206066
- date added to LUP
- 2025-07-07 09:31:59
- date last changed
- 2025-07-07 09:31:59
@misc{9206066,
abstract = {{This study investigated how acoustical challenges, specifically changes in signal-to-noise ratio (SNR), affect the neural tracking of attended and ignored speech in individuals with hearing impairment. The analysis began with the speech envelope and was progressively extended by incorporating phonetic information as well as outputs from all layers of OpenAI’s Whisper model. The goal was to gain a deeper understanding of the brains’ audio processing pathways during Cocktail Party Problem like situations. Using a Temporal Respons Function to model brain activity, the results showed that adding these features improved the model’s ability to predict neural responses, with the correlation between predicted and recorded EEG increasing from 3.2% when using only the envelope to 3.8% when combining the envelope, phonemes, and Whisper’s layer-wise outputs (p = 0.0021). When distinguishing neural responses between attended and ignored speech, the inclusion of these additional features did not influence the classification accuracy significantly compared to models using only basic acoustic features.
This work has potential applications in hearing technology, enabling the assessment of hearing and the effects of various hearing devices. Furthermore, it could inform signal processing algorithms in hearing aids, enhancing speech comprehension tracking and improving auditory attention by tailoring hearing devices to the specific needs of individual users.}},
author = {{Müller, Josefine and Thulin, Cajsa}},
language = {{eng}},
note = {{Student Paper}},
title = {{Acoustic Degradation Affects Neural Measures of Speech Tracking and Understanding}},
year = {{2025}},
}