11:00 - 12:30
Parallel sessions 8
11:00 - 12:30
Room: HSZ - N5
Chair/s:
Fritz Günther, Markus Kiefer
Embodied and grounded cognition approaches have remained enduring focal points in cognitive psychology. Although some subtle differences are sometimes postulated, both approaches converge on the assumption that cognition is essentially based on a reinstatement of processes of perception, action and introspection. How exactly the symbols that are central our higher cognition and communication, such as linguistic forms and abstracted mental representations, obtain their meaning from sensorimotor experience is one of the open challenges in this line of research. Highly interesting experimental-behavioural studies have been conducted that produced important insights. At the same time, we are experiencing the theoretical and empirical limits of this approach:
On the one hand, research on embodied cognition is often missing the formalisation, quantification, and precision required to make theoretically substantive advances – a gap to be filled with computational modelling. Here, recent work has brought forward large-scale data-driven representation models built from different data sources, such as language and vision. These allow us to exactly operationalize to what extent information from different modalities of experience shape our semantic representations, and investigate their specific influences on cognitive processes.
On the other hand, theories of embodied cognition ultimately always result in claims about processes in specific cognitive systems (shared between higher cognition and sensorimotor or introspective processing), which are hard to evaluate with purely behavioural approaches and instead require neuroscientific methods. This includes electrophysiological methods with high temporal precision, as well neuroimaging methods with high spatial resolution; together, these techniques allow us to precisely map neural processes that underpin higher cognition.
 In this symposium, we bring together recent advances integrating computational and neuroscientific approaches to embodiment research: Computational models yield precise predictions at the system level, which in turn can be tested with neuroscientific methods. The presentations in this symposium highlight the advantage of an interplay of computational and neuroscientific approaches for various fields of embodied cognition such as language, memory and semantics.
Submission 113
Indirect Experiential Grounding: Semantic Similarity of Abstract Scientific Concepts Can Be Decoded from Activity Patterns in Visual and Motor Cortex
SymposiumTalk-03
Presented by: Markus Kiefer
Markus Kiefer 1, Marcel Harpaintner 1, Natalie Trumpp 1, Alexander Berger 1, Fritz Günther 2, Martin Ulrich 1
1 University of Ulm, Germany
2 Humboldt-University, Berlin, Germany
Grounded cognition theories propose that interactions with situations establish experiential memory traces that constitute conceptual meaning. However, as abstract scientific concepts are frequently learned through language, a proposed indirect grounding mechanism postulates that modal representations are extrapolated from distributional language-based representations, which are themselves mapped onto existing modal representations. Using functional magnetic resonance imaging (n = 51), we tested the mapping of distributed language representations on modal representations by asking whether semantic similarity of distributional language representations of scientific concepts corresponds to the similarity of neural activation patterns in modal brain systems. Semantic similarity based on both distributed language representations and experiential features was decoded from the multi-voxel activity pattern within occipital and fronto-parietal cortex, overlapping with activation induced by real visual perception and hand action, in addition to multimodal areas outside the brain regions sensitive to perception or action. Although its functional relevance for extrapolating modal representations remains to be determined, this mapping between language and the visuo-motor system is a fundamental element of the indirect grounding mechanism. This mechanism provides sensory-motor experience for the unseen and enriches knowledge for concepts learned exclusively from language.