In humans, the ability to visually categorize faces (i.e., discriminate faces from other objects and generalize this discrimination across individual faces) follows a protracted development during the first year of life. It was recently shown that this developing visual ability is boosted by an odor in 4-month-old infants, neural face categorization being strongly enhanced in the context of mother’s body odor. Whether this influence operates until the end of the first year, as face categorization becomes more efficient by itself, must be established. Here, we recorded the electroencephalogram (EEG) of 4- to 12-month-old infants (N = 50) while they were watching streams of rapidly changing pictures (6 pictures/sec leading to a 6-Hz frequency of stimulation) including living and non-living objects. Human faces were periodically inserted once per second (i.e., at 1 Hz). During visual stimulation, infants were also exposed to a T-shirt imbued with maternal odor vs. an unworn, baseline T-shirt. Using a frequency-tagging approach, we reveal that the amplitude of the face-selective neural response tagged at 1 Hz in the EEG spectrum increases with age over the occipito-temporal cortex, marking the development of face categorization. Critically, while the strength of the face-selective response increases as a function of age, the “facilitative” effect of maternal odor decreases over the same time. These results suggest the operation of a developmental trade-off between vision and olfaction and support the view that visual perception relies on odor cues in developing infants until the sole visual system becomes able on its own to readily achieve categorization.