Neural Speech Tracking in EEG: Integrating Acoustics and Linguistics for Hearing Aid Users
(2024)Department of Automatic Control
- Abstract
- This master thesis explores the neural encoding of speech features for hearing aid users. The study utilizes electroencephalography (EEG) and audio data from an experiment that stimulates a Cocktail Party scenario. This is a complex auditory scene, especially difficult for individuals with hearing impairments. The primary objective of the study is to investigate how different acoustic and linguistic speech features are represented in the brain response and how these representations are influenced by hearing aid settings. The speech features analyzed are the acoustic envelope, phonetic features, word onset, and word surprisal. The word surprisal values were derived from GPT-2. Temporal Response Functions (TRFs) and multivariate TRFs (mTRFs)... (More)
- This master thesis explores the neural encoding of speech features for hearing aid users. The study utilizes electroencephalography (EEG) and audio data from an experiment that stimulates a Cocktail Party scenario. This is a complex auditory scene, especially difficult for individuals with hearing impairments. The primary objective of the study is to investigate how different acoustic and linguistic speech features are represented in the brain response and how these representations are influenced by hearing aid settings. The speech features analyzed are the acoustic envelope, phonetic features, word onset, and word surprisal. The word surprisal values were derived from GPT-2. Temporal Response Functions (TRFs) and multivariate TRFs (mTRFs) were employed to examine the correlation between these features and EEG signals during speech processing in both attended (target) and unattended (masker) speech scenarios.
The TRFs were estimated by training a forward model using a boosting algorithm. In this process, the speech features serve as the predictor variables (X-data), and the EEG signals serve as the response variables (Y-data). The boosting algorithm iteratively improves the model by combining multiple weak learners to better predict the EEG responses based on the given speech features.
The study found that target and masker speech are significantly distinguishable using TRF models trained on these features. It also revealed that hearing aid conditions impact their encoding. Among the features analyzed, the acoustic envelope had the highest correlation with neural responses. Adding other predictor variables to the Envelope model did not improve the correlation. Further, all speech features were found to have unique neural encoding. The acoustic envelope and phonetic features could be correlated to early processing, while word onset and word surprisal are reflected in later neural responses.
Our findings suggest that speech features are important in understanding how hearing aid users process speech, which could lead to future development of hearing aids that are not only fitted to the user’s needs but also tailored to their unique neural responses. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9174277
- author
- Almgren, Klara and Mentzer, Annie
- supervisor
- organization
- year
- 2024
- type
- H3 - Professional qualifications (4 Years - )
- subject
- report number
- TFRT-6243
- other publication id
- 0280-5316
- language
- English
- id
- 9174277
- date added to LUP
- 2024-09-13 13:20:53
- date last changed
- 2024-09-13 13:20:53
@misc{9174277,
abstract = {{This master thesis explores the neural encoding of speech features for hearing aid users. The study utilizes electroencephalography (EEG) and audio data from an experiment that stimulates a Cocktail Party scenario. This is a complex auditory scene, especially difficult for individuals with hearing impairments. The primary objective of the study is to investigate how different acoustic and linguistic speech features are represented in the brain response and how these representations are influenced by hearing aid settings. The speech features analyzed are the acoustic envelope, phonetic features, word onset, and word surprisal. The word surprisal values were derived from GPT-2. Temporal Response Functions (TRFs) and multivariate TRFs (mTRFs) were employed to examine the correlation between these features and EEG signals during speech processing in both attended (target) and unattended (masker) speech scenarios.
The TRFs were estimated by training a forward model using a boosting algorithm. In this process, the speech features serve as the predictor variables (X-data), and the EEG signals serve as the response variables (Y-data). The boosting algorithm iteratively improves the model by combining multiple weak learners to better predict the EEG responses based on the given speech features.
The study found that target and masker speech are significantly distinguishable using TRF models trained on these features. It also revealed that hearing aid conditions impact their encoding. Among the features analyzed, the acoustic envelope had the highest correlation with neural responses. Adding other predictor variables to the Envelope model did not improve the correlation. Further, all speech features were found to have unique neural encoding. The acoustic envelope and phonetic features could be correlated to early processing, while word onset and word surprisal are reflected in later neural responses.
Our findings suggest that speech features are important in understanding how hearing aid users process speech, which could lead to future development of hearing aids that are not only fitted to the user’s needs but also tailored to their unique neural responses.}},
author = {{Almgren, Klara and Mentzer, Annie}},
language = {{eng}},
note = {{Student Paper}},
title = {{Neural Speech Tracking in EEG: Integrating Acoustics and Linguistics for Hearing Aid Users}},
year = {{2024}},
}