Prominent explanations of anti-immigrant attitudes give centrality to socio-cultural differences between immigrants and host communities and to the economic threats posed by newcomers. What happens when incoming immigrants constitute little economic threat and are culturally similar to the host community? In this paper, we provide evidence that backlash to immigrants can occur even when socio-cultural differences and economic threats posed by immigrants are minimal. We do so using the case of Switzerland, which opened its borders and labor market to the rest of Europe in the 2000s. Using a difference-in-differences approach, we first show that the number of immigrants living and working in Swiss border municipalities increased dramatically after the borders opened, with the vast majority of these immigrants coming from neighboring France, Germany and Italy. In the same municipalities, we find that support for anti-immigrant parties increased by approximately five percentage points after borders were opened. These findings contribute to our theoretical understanding of anti-immigrant sentiment and have policy implications for the addressing the political consequences of the European Union’s principle of free movement.