11:00 - 12:30
Parallel sessions 8
11:00 - 12:30
Room: HSZ - N5
Chair/s:
Fritz Günther, Markus Kiefer
Embodied and grounded cognition approaches have remained enduring focal points in cognitive psychology. Although some subtle differences are sometimes postulated, both approaches converge on the assumption that cognition is essentially based on a reinstatement of processes of perception, action and introspection. How exactly the symbols that are central our higher cognition and communication, such as linguistic forms and abstracted mental representations, obtain their meaning from sensorimotor experience is one of the open challenges in this line of research. Highly interesting experimental-behavioural studies have been conducted that produced important insights. At the same time, we are experiencing the theoretical and empirical limits of this approach:
On the one hand, research on embodied cognition is often missing the formalisation, quantification, and precision required to make theoretically substantive advances – a gap to be filled with computational modelling. Here, recent work has brought forward large-scale data-driven representation models built from different data sources, such as language and vision. These allow us to exactly operationalize to what extent information from different modalities of experience shape our semantic representations, and investigate their specific influences on cognitive processes.
On the other hand, theories of embodied cognition ultimately always result in claims about processes in specific cognitive systems (shared between higher cognition and sensorimotor or introspective processing), which are hard to evaluate with purely behavioural approaches and instead require neuroscientific methods. This includes electrophysiological methods with high temporal precision, as well neuroimaging methods with high spatial resolution; together, these techniques allow us to precisely map neural processes that underpin higher cognition.
 In this symposium, we bring together recent advances integrating computational and neuroscientific approaches to embodiment research: Computational models yield precise predictions at the system level, which in turn can be tested with neuroscientific methods. The presentations in this symposium highlight the advantage of an interplay of computational and neuroscientific approaches for various fields of embodied cognition such as language, memory and semantics.
Submission 663
Predictive Gestures: An EEG Study of Multimodal Affirmation and Negation
SymposiumTalk-04
Presented by: Samuel Sonntag
Samuel Sonntag 1, Merle Weicker 2, Ian Mackenzie 1, Carolin Dudschig 1
1 University of Tübingen, Germany
2 University of Frankfurt, Germany

Human face-to-face communication encompasses far more than spoken words; it involves a rich combination of multimodal signals such as speech, gestures, facial expressions, and more. Despite this ubiquity of multimodality in communication, the precise role that gestures play in shaping understanding remains to be fully determined. Motivated by experimental evidence suggesting that multimodal information can facilitate comprehension under increased processing demands, we conducted an initial study with an experimental paradigm adapted from basic conflict tasks in cognitive psychology. We manipulated the compatibility of gestural (head shakes/nods & thumbs up/down) and verbal information (yes/no) and investigated whether negation - a linguistic universal that has proven to elicit increased processing demands - would specifically benefit from the presence of multimodal information. Interestingly, we did not find convincing evidence to support this multimodality for compensation account across several behavioral experiments. In contrast, affirmative information seemed to benefit more from the presence of multimodality than negative information. In follow-up studies, we aim to further delineate the integration of multimodal information, with particular emphasis on the function and role of gestures in speech processing and their contribution to predictive processing. The results of these studies - including electrophysiological evidence - will be discussed.