The illusory truth effect describes the effect of repetition on judgments of truth, where previously encountered statements are judged to be more valid than new statements. One line of research has focused on individual differences in this effect: Examining how the truth effect covaries with dispositional variables allows researchers to infer underlying mechanisms and, ultimately, to design interventions against the effect. However, findings regarding individual differences have been inconsistent: While some studies find correlations with certain dispositional variables, others do not. We argue that these ambiguities may be due to poor psychometric properties of truth effect experiments. Specifically, correlations may be attenuated by low reliability, so that some correlations may not even be recoverable. Therefore, in a large-scale reanalysis of multiple published data sets, we assess the psychometric goodness of truth effect experiments using a signal-to-noise ratio introduced by Rouder et al. (2023) that quantifies task goodness independent of trial size. Our analysis shows that individual truth effect scores typically have low reliability, potentially masking true correlations with other variables. We identify experimental characteristics that promote reliability, such as stimulus material, initial task conditions, sample composition, and study implementation (online vs. lab). Our findings provide guidance for more reliable study designs when investigating individual differences in the truth effect and demonstrate that achieving high reliability is feasible. This work thus provides the basis for robust research on individual differences in the truth effect.