In many everyday situations, reading or listening tasks are difficult to process on a perceptual level as examples such as illegible handwriting or connection problems during video conferencing demonstrate. It is yet unclear whether the presence of such perceptual noise influences people’s capability to focus their attention on a current task. Therefore, in the present study, we investigated whether perceptual noise imposed while reading or listening to a Sherlock Holmes story affects mind-wandering processes as well as text comprehension. We presented 175 participants with the story in either a visual or an auditory presentation format. For half of the participants in each presentation-format condition, the story was superimposed by (visual or auditory) Gaussian noise. All participants’ thoughts and motivational states were probed from time to time during reading or listening. Independent of the presentation format, we found that the participants in the noise conditions reported higher mind-wandering levels and performed worse in a later comprehension test compared to the participants in the no-noise conditions. These negative effects of perceptual noise on on-task focus and comprehension were partly driven by motivational factors: The presence of noise decreased reading/listening motivation which, in turn, increased mind wandering. Our results are well in line with results from studies investigating the effects of increased semantic reading difficulty on mind-wandering, indicating that semantic and perceptual difficulty might elicit similar processes.