With social and political science an increasingly international enterprise (Wilson & Knutsen, 2020), a growing number of studies published in leading journals investigate (political) phenomena occurring in countries where English is not the main language. Following other types of comparative research, it is no surprise then that studies using computational text analysis methods (CTAM) are increasingly conducted in an international, inter-lingually comparative fashion (e.g., Lind et al., 2019). However, this rapid internationalization is hardly reflected in the development of CTAM. Owing to the considerable head start of English-language computational development—both due to early U.S. dominance in computer technology and due to the specific role of English as scientific lingua franca—many resources, tools, and training required for cutting-edge CTAM development are available only, or in far better quality, for English-language text analysis. This present study delves deeper into the relationship between languages and the use of CTAM methods, utilizing a novel dataset derived from a pre-registered survey of 421 social science researchers. Specifically, we examine how the language of analysed texts influences researchers’ validity concerns about the use of CTAM methods. We also examine what motivations prevent researchers from examining multilingual textual corpora. Our results will inform the debates on strengthening the link between CTAM tool development on the one hand and social science research on the other.