The Spatial Numerical Association of Response Codes (SNARC) effect refers to the observation that relatively small (e.g., 1) and large numbers (e.g., 9) elicit faster left-sided and right-sided manual responses, respectively. In a variation known as the attentional SNARC effect, merely looking at numbers caused a left- or rightward shift in covert spatial attention, depending on the number’s magnitude. In our study, we probed the notion that numbers induce shifts of spatial attention in accordance with their position on a mental number line (MNL). Critically, we removed any putative spatial response code that may contaminate the responses. We used a square and a tilted square as targets, thereby situating the decisive response dimension in the ventral, non-spatial processing stream. In two experiments where numbers were used as non-informative cues preceding a temporal order judgment (TOJ) task, we did not observe a deflection of the locus of spatial attention as a function of the numerical magnitude of the cue. In a third experiment, finding a significant modulation of TOJ performance as a function of the pointing direction of arrow cues allowed us to rule out the possibility that the absence of any significant modulation in experiments 1 and 2 was due to a lack of sensitivity of our task set-up. We conclude from the current findings that the spatial codes that the perception and naming of numbers potentially elicit are not in and by themselves sufficient to elicit deflections of spatial attention.