The N-back task is a typical experimental paradigm for measuring working memory (WM). Introduced by Kirchner (1958), the N-back task is the most dominant measure of WM and its neurological substrates. It is widely discussed in neuroimaging studies with little emphasis on behavioral results (Conway et al., 2005). Studies mainly use classical settings, i.e., participants react to stimuli on a computer monitor by pressing keys on a keyboard. In these tasks, stimuli such as digits or letters are typically used, while response accuracy and response times are indicators of the level of difficulty of the task. We recreate this task in Virtual Reality (VR), where we collect 3D movement trajectories of participants instead of only keypresses, tracking participants’ reactions in real-time. We then compare performance in this adapted task to that of a corresponding task performed using a keyboard and a computer screen. We report the general method for this adaptation of the N-back task in VR with the goal of providing a behavioral index that can combine well with physiological data. Our results show that this method can be used for many adaptations of memory paradigms and provides a wealth of information about the time course of cognitive processing that is usually obtained with physiological measures. Moreover, it can combine well with these measures, providing opportunities for future research, especially with VR technology.