Speech and language research, such as auditory attention decoding (AAD) to speech streams, has traditionally relied on scripted stimuli from professional speakers, often involving passive listening tasks. We investigate audiovisual speech tracking and auditory attention using natural, unscripted stimuli to reflect natural listening demands better. We use electroencephalography (EEG) and other physiological measures in telepresence and virtual listening scenarios to investigate cortical speech tracking derived from the correlation of scalp EEG and speech envelope time series data.
We will report the results of several studies. In one study we used unscripted natural speech stimuli from six speakers and found clear evidence for cortical speech tracking. Interestingly, specific speech features, such as pitch variability and rhythm, can significantly enhance the magnitude of cortical speech tracking. The subsequent analysis included likeability ratings of the different speakers. Significant correlations between liking and cortical speech tracking suggest a link between subjective preferences and attentional processing of speech. Our work demonstrates the value of EEG-based cortical speech tracking in virtual environments to understand what factors influence listening and how listeners cope with natural variations in communication scenarios.