Brain responses elicited during action observation have been interpreted as reflections of spontaneus predictive processes. It is assumed that such predictions have a multi-level structure, especially if the observed actions are complex, as is usually the case in everyday life. In a series of fMRI studies, we addressed the question of whether episodes can function as higher-order prediction models during action observation. Subjects encoded actions as short episodes that were later retrieved using partially manipulated cues. The behavioral and neural effects of episodic prediction and episodic prediction errors are discussed. Findings suggest that cortical midline and hippocampal structures may provide higher-order input to lower-level cortical areas in everyday action observation.