The traditional analysis approach in cognitive psychology aggregates behavioral measures as response time (RTs) and error rates (ERs) across all trials of a particular condition occurring in the experimental session. Implicitly, this approach assumes that cognitive processing is stationary during the experimental session. This assumption, however, is questioned by observations of practice-induced changes in experimental effects. Here, we present a new method, the effect course analysis, to study practiced-induced effect changes during an experimental session. The effect course analysis is a model-free, non-parametric method designed to reveal effect changes within an experimental session on a continuous time scale. This method is based on cluster-based permutation testing, which is widely used in the analysis of neuro-physiological data. The effect course analysis depicts the time course of an experimental effect by contrasting smoothed RTs / ERs of two experimental conditions along the trials of the experiment and testing at which point in the experiment it is unlikely to obtain such an effect (difference between conditions) by chance using cluster-based permutation tests.
By re-analyzing previously published data sets, we show that experimental effects may not be static, but change during the course of an experimental session. Hence, if one cannot be certain that cognitive processing is stationary; investigating the time course of experimental effects appears an intriguing approach to determine how cognitive processing is influenced by practice.