The uncanny side of AI: cognitive biases in perception of ChatGPT's theory-of-mind abilities
Mon-Main hall - Z3-Poster 1-2707
Presented by: Tobias Senger
Our perception of an artificial intelligence's (AI) social abilities significantly impacts our interactions with these systems. Here we investigate whether people attribute Theory-of-Mind (ToM) abilities to ChatGPT, and when ChatGPT’s ToM ability is considered creepy. Using the Mind-Perception Hypothesis and the Uncanny Valley phenomenon, we explore biases in human perception of ChatGPT's abilities. In an online study, 28 participants (62% female) rated written responses to scenarios assessing ToM reasoning from the Adult Theory of Mind Test (A-ToM). Responses of high and medium ToM abilities were generated by ChatGPT according to the A-ToM scoring system, and were presented as either human or ChatGPT-generated. We tested whether perceived ToM abilities and perceived creepiness varied based on the assumed authorship (Human vs. ChatGPT) and A-ToM level (Low vs. High). Preliminary results show that people attribute higher ToM abilities to ChatGPT when responses demonstrate a profound understanding of social scenarios (high A-ToM level), compared to responses with a low A-ToM level. Yet, the same responses were rated as equally competent, regardless of ToM ability, when attributed to a human. Consistent with the Mind-Perception Hypothesis, ChatGPT abilities were perceived as more creepy when responses indicated high ToM abilities, compared to both ChatGPT responses with low ToM ability, and to supposedly human responses, regardless of ToM ability. Negative attitudes towards AI had no direct influence on response ratings, but interacted with attributed authorship in perceived creepiness. Our findings suggest that humans are more critical when assessing ChatGPT’s skills compared to when assessing those of a human.
Keywords: