Control at our fingertips? Co-Acting with Avatar Hands and the Role of Intentionality during Virtual Joint Action
Wed-B21-Talk VII-02
Presented by: Luke Bölling
The occurrence of the joint Simon-Effect (JSE) can be regarded as an indirect measurement of self-other integration in dyadic interactions. Current JSE research hints towards a prominent role of the co-actor’s agency and/or intentionality without fully differentiate between bottom-up and top-down manipulations. To separate bottom-up and top-down modulation effects, we conducted a series of three browser-based Joint Simon experiments (N ≈ 300). Firstly, we established a new digital scenario to reliable evoke a JSE. A virtual human hand was presented on the right side controlled by the participant. By performing a go/nogo-task with (joint) or without (individual) a second computer-controlled hand on the left side of the display, we showed a reliable JSE based on reaction times, and we varied the appearance of the virtual co-actor's hand between subjects, showing a stronger JSE for a human compared to a robot co-actor (E1). Additionally, the perceived agency of the co-acting hand was manipulated solely using visual information (bottom-up) by comparing a moving and a non-moving co-actor in the second experiment. We found no effect for the agency-manipulation, but a persistent bottom-up-effect of the co-actor’s hand (human vs. robot) indicating a prominent role of bottom-up-information regarding a virtual co-actor’s intentionality. In Experiment 3, we showed an increased JSE for an intentional described (top-down) virtual robot as compared to a non-intentional virtual robot. Conclusively, this series suggests a model in which bottom-up and top-down information need to be integrated to fully understand the role of intentionality in joint action.
Keywords: joint Simon effect, stimulus-response compatibility, agency, intentionality, virtual co-actor