Litter in urban areas is an unnecessary burden to municipal budgets and is done through both intentional and unintentional means. Past research tells us that the presence of litter on streets encourages further littering behaviours. If so, regular street cleansings would reduce the deposition rate of litter. Litter is a direct cost to municipalities and can also be seen to cause loss of esthetic and moral values to its citizens and visitors. Here we search for optimal strategies for when to clean urban areas from litter in order to minimise the total costs of litter to a municipality. The cleaning and deposition of litter is formulated as a Markov Decision Process. We use chewing gum as an indicator for litter. Optimal strategies are represented with different budget constraints.