Perceptual load theory proposes that processing of distractor stimuli is determined by the level of perceptual load (Lavie, Hirst, De Fockert & Viding, 2004). We conducted a study with female and male names as targets, the presentation of neutral faces as distractors, and a variation of perceptual load. In a comparable study, He and Chen (2010) observed the disappearance of the interference effect of faces at high perceptual load, in line with perceptual, load theory. However, Lavie (2003) earlier found that even for high perceptual load flanker effects for (famous) faces were found. In our first version of the study with neutral expression faces participants categorized names that were conventionally female or male, into their respective genders. The target was presented with 1, 3, or 5 pseudo-names (i.e., the variation of perceptual load). A distractor male or female neutral face was presented either to the right or left of the name and pseudonyms, creating either congruent or incongruent trials. Our results showed a congruency effect regardless of the set size, in line with Lavie’s earlier result and in contrast to He and Chen. We discuss these findings with regards to an adaptive and social necessity for processing all faces, even if they are unfamiliar and irrelevant to the task at hand.