When assessing individual differences in experimental psychology, we often ignore the hierarchical structure of the data. This omission may lead to unstable person-level estimates and attenuated correlations. To address these issues, Bayesian hierarchical modelling has been proposed to account for sample noise on the trial level and to improve estimation of correlations between experimental tasks. We present a Bayesian hierarchical workflow for individual differences that aims at guiding experimental psychologists in optimal measurement of person-level parameter and correlation estimates. The first part of the workflow covers applying existing cognitive or psychometric models, the second part outlines steps for developing new models. Based on previous Bayesian workflows, we describe key steps such as prior predictive checks, computational faithfulness, model sensitivity, parameter recovery, and posterior predictive checks. The new focus is on the estimation of between-subject variability and correlations which comes with its own set of challenges.

Prior prediction is used to assess whether a model can produce sensible group- and individual patterns, ensuring that the model’s assumptions allow realistic between-subject variability. Computational faithfulness ensures correct posterior sampling and involves checking computational diagnostics. Model sensitivity, the Bayesian version of power analysis, checks if sample sizes (number of participants and trials) are sufficient to avoid high uncertainty about parameter estimates. Parameter recovery tests whether model parameters are identifiable and reliable. Finally, posterior predictive checks evaluate how well the model fits data and reproduces observed patterns. We provide a template with reproducible reports to enhance transparency and robustness in hierarchical cognitive modeling analyses.