Laterality judgments made according to a third-person perspective are easier when this perspective aligns with the perspective of the participant than when the two perspectives are misaligned. This alignment advantage has been attributed to mental perspective transformation (assumed to take place before the judgment is made). In standard perspective taking tasks, alignment of perspectives is confounded, however, with spatial (i.e., left-right) correspondence of the target stimulus feature and the response. To deconfound these factors, we modified the Own Body Transformation Task, which requires laterality judgments of a stimulus feature according to the perspective of a human-like stimulus figure (presented in upright format), by asking participants to give “same-different” rather than “left-right” responses. Specifically, participants judged whether laterality of the feature matched or mismatched the laterality of the feature on a prime figure presented before (i.e., sequential comparison task by pressing a left- vs. a right-sided key. When the target figure was presented in back-facing orientation, a regular Simon effect occurred (i.e., responses were facilitated when the side of the required response corresponded to the side of the target feature). When the stimulus figure was presented in front-facing orientation, however, response performance was better when the laterality of the target feature judged from the figure’s perspective matched (and thus, judged from the participant’s perspective, mismatched) the side of the required response. This reversal of the Simon effect suggests that spatial stimulus-response compatibility is dominated by task-defined semantics (i.e., laterality coding according to a third-person perspective).