Previous studies using representational similarity analysis (RSA) of fMRI data demonstrated that activation patterns in the lateral occipito-temporal cortex (LOTC) during action observation capture the semantic similarity structure of actions, in line with the view that the LOTC contributes to action understanding. Here we aimed to investigate the time course of neural representations of low-, mid- and high-level action features.
We recorded EEG from 24 participants in a delayed matching task. Stimuli consisted of static images spanning 27 action categories of everyday actions (e.g. riding a bike, drinking coffee), shown in different combinations of actors, locations and viewpoints. Participants were asked to match stimuli on action category, actor, or location.
Separately for each time point we computed representational dissimilarity matrices (RDMs) pertaining to neural representations of action categories from time-resolved multivariate pattern analysis of EEG data. These neural RDMs were correlated to independently generated behavioural RDMs corresponding to semantic action (dis)similarities, revealing a time course of neural semantic action representations. In addition, we determined RDMs to capture the similarity of movement type, body part, scenes, and objects, revealing the respective neural representations over time.
We observed that semantic organizing principles observed in the LOTC can be observed from approximately 150 ms post stimulus presentation onwards, whereas organizing principles related to action context and kinematics can already be observed earlier. These outcomes suggest that neural representations pertaining to low- and mid-level action features precede those of semantic action features, capturing the computational steps that allow humans to recognize observed actions.