Prioritized processing of fearful compared to neutral faces is reflected in differential hemodynamic responses and event-related potentials (ERPs). A famous fMRI study by Pessoa and colleagues (2002) reported an abolition of emotional differentiation under distraction tasks. However, there is an ongoing debate over the extent to which emotional faces can be processed without attention or under competing high-load conditions, as early ERPs, such as the N170, are reliably modulated by fearful facial expressions under high perceptual load and even in studies using efficient masking methods. Here, we replicated the set-up from Pessoa and colleagues (2002) in a combined EEG/fMRI study with a large sample (N = 51). In three tasks, participants paid attention to the gender of the face and performed either an easy color discrimination task or a difficult orientation discrimination task, while fearful and neutral facial expressions were presented. Fearful faces elicited increased N170 responses during all three tasks, accompanied by increased BOLD responses in the face network (fusiform gyrus, superior temporal sulcus), while amygdala responses did not differentiate between fearful and neutral faces. ERP and fMRI measures did not interact with the attention task. These findings reveal that the early stages of fearful face processing appear to be invariant to manipulations of load and/or competing attention tasks.