When humans act, they produce perceptual changes through motor activity. Doing so typically goes along with the experience of agency over the produced perceptual change. Here we investigated how the spatial relationship between motor activity and ensuing perceptual changes shape agency experience and memory for the produced event. Participants engaged in a task where they responded to a visual stimulus—a white box—by executing self-decided up or down key responses. Subsequently, the box moved either spatially congruently or incongruently to the participants' response. After the box stopped moving, a word was revealed within it. A recognition memory test for these words was conducted later. After each trial, participants rated their sense of agency. In one condition, trial order was random, and thus congruent and incongruent trials were presented intermixed, whereas they were presented in blocked sequences in another condition. Results showed a higher sense of agency following congruent as compared to incongruent trials, and a higher sense of agency given a blocked rather than mixed trial order. Recognition memory was better for words occurring in congruent trials as compared to incongruent trials, with no effect of trial order. However, this effect was small and only reliably found when collapsing across the different trial order conditions. An additional investigation of effects of changes in control between two consecutive trials yielded no effect of control change on memory. These findings highlight the impact of action-effect congruency on perceived control and suggest its potentially facilitating effect on memory.