Past research has shown that the arousal level of emotional faces as targets in a visual search paradigm is a more influential factor on performance than the emotional category of the face (e.g., angry, happy). These studies did not address the effect of facial stimuli on executive attentional processing (i.e., inhibition), which is a crucial factor in attentional biases. Understanding attentional biases to emotional faces has clinical implications for social anxiety, as they play a vital role in the acquisition and maintenance of the disorder. The goal of the present study was to test the effects of task-irrelevant emotional arousal elicited by emotional faces on visual search performance. In Experiment 1, facial expressions acted as distractors presented throughout the task. In Experiment 2, faces were presented only before the task. We found that the level of arousal elicited by faces can better describe their effects on visual search performance compared to their discrete emotional categories. In Experiment 1, faces with higher arousal ratings were more distracting, possibly because active inhibition is necessary to solve the task. However, in Experiment 2, higher levels of arousal resulted in better performance, possibly because they activate the arousal system, initiating readiness for action. We also showed that social anxiety diminishes attentional performance due to its influence on executive processing. This does not support the involvement of bottom-up attentional biases in the maintenance of social anxiety that many interventions seem to target.