Can AI Capture Cultural Differences? Comparing Large Language Models in China and the United States
P5-S125-1
Presented by: Chao-Yo Cheng
This study explores whether generative AI (GenAI) can capture cultural differences. By comparing the performance of different commercial large language models (LLMs) in China and the United States, we examine the potential of LLMs as a useful tool to describe or even explain the prevalence of collectivism in China as well as its regional variation. Through a variety of prompt engineering, more specifically, we ask GenAI to simulate responses that may resemble patterns well noted by prior research. Our results present a mixed picture. On the one hand, preliminary results based on Chinese LLMs indicate that GenAI may not be able to capture the regional differences in collectivist values across China as expected. On the other hand, we find that Chinese LLMs can produce results largely in line with existing studies when it comes to the factors associated with the presence of collectivism under various historical and socioeconomic contexts, such as grain production in contemporary China. Our findings demonstrate that GenAI can serve as an effective tool for exploratory research, enriching the toolkit for research in politically challenging contexts where typical survey projects may not be feasible due to various ethical and logistics constraints.
Keywords: Large Language Models, GenAI, simulation, collectivism, China