08:30 - 10:00
Mon—HZ_7—Talks1—2
Mon-Talks1
Room:
Room: HZ_7
Chair/s:
Carolin Dudschig, Yvonne Portele
Redundant Gestures: Integrating Head Nods, Shakes, Thumbs Gestures and Speech
Mon—HZ_7—Talks1—204
Presented by: Samuel Sonntag
Samuel Sonntag 1*Ian Mackenzie 1Emanuel Schütt 1Merle Weicker 2Sara Oguz 2Carolin Dudschig 1
1 Eberhard-Karls-Universität Tübingen, 2 Goethe-Universität Frankfurt
We communicate not only through words but also in a rich, multimodal way, combining verbal and non-verbal signals. This holds for language in general, but is particularly interesting for negation for several reasons. For example, there is a wide variety of non-verbal signals to transfer negation. These non-verbal negation markers can be face- (e.g., the not face), head- (e.g., head shake) or hand related (e.g., thumbs down). Given the rich multimodal nature of negation, it makes it an ideal testing ground to investigate how the brain integrates simultaneous information from multiple sensory inputs. Using experimental paradigms adapted from basic conflict tasks in cognitive psychology - we investigated the integration of verbal and non-verbal negation markers. Earlier studies showed that standard conflict effects are present in the processing domain: Response times are faster if the verbal and non-verbal information matches (e.g. “no” combined with a “thumbs down” gesture) than when they mismatch (e.g., “no” combined with a “thumbs up” gesture). Interestingly, this effect was stronger for head-related gestures (e.g., head shake) than for hand related gestures (e.g., thumbs down). Also, these compatibility effects were stronger for affirmative than negated combinations. In several follow-up studies, we now analyze the origin of these compatibility effects via the addition of uni-modal conditions that allowed us to investigate facilitation versus inhibitory sources of these effects. The results of these studies will be discussed.
Keywords: Neagtion, gesture, non-verbal communication, conflict processing