Submission 705
Components of Creativity: Language Model-Based Predictors for Clustering and Switching in Verbal Fluency
SymposiumTalk-05
Presented by: Özge Alacam
This work investigates the psychometric capacities of Language Models (LMs) in the verbal fluency task, an experimental paradigm used to examine human knowledge retrieval, cognitive performance and creative abilities. We focus on switching and clustering patterns and seek evidence to substantiate them as two distinct and separable components of lexical retrieval processes in LMs. Specifically, we prompt different transformer-based LMs with verbal fluency items and ask whether metrics derived from the language models’ prediction probabilities or internal attention distributions offer reliable predictors of switching/clustering behaviors in verbal fluency. We find that token probabilities, but especially attention-based metrics have strong statistical power when separating between cases of switching and clustering.