Free listing is a simple and effective, semi-quantitative method primarily used in cognitive anthropology and cultural studies. It is a type of restricted continuous association task in which participants list as many items of a domain as possible (for example, participants may be asked to list all adjectives which can be used to describe the aesthetics of music). Whereas in psychology and psycholinguistics similar tasks (the semantic verbal fluency task or the property listing/ feature production task) are used with the aim to study memory- and language-related processes (and their pathologies), free listing is used to gain insight into the vocabulary people use for a given domain and its conceptual structure. To examine the reliability of the method, i.e., how well the results were replicated at the aggregate level and for individuals, we collected data on the music domain from different samples (of one sample twice). This resulted in an ordered set of five data sets, with the samples being successively more similar in terms of language, time of study, place of study, and participants. We computed similarities between the datasets in terms of their features, as well as their term-frequency distributions. The replication of the most important terms across all samples as well as a significant positive correlation between the two types of similarity indicate that the free listing method yields reliable results.