Does elite polarization lead to a decline in electoral accountability? Recent research has found that ideological polarization is usually driven by elites, only later spreading to the public at large. In this paper, we argue that elite polarization reduces electoral accountability even when levels of mass and affective polarization do not change. As the candidates’ policy platforms diverge, voters increasingly forgo accountability because the electoral importance of valence issues decreases, including for matters such as competence, effective governance, corruption and economic outcomes. Original experimental evidence from three countries (United States, United Kingdom, Spain, total N = 4,200), in which we manipulate policy polarization and partisan identities separately, confirms that higher levels of candidate polarization cause reductions in accountability-driven voting. Qualitative evidence from voters’ own words shows that cognitive, in-group or partisan biases, such as partisan motivated reasoning or blind partisanship, are not necessary to explain declines in electoral accountability. Instead, many voters rationally forgo accountability in polarized contexts because they are aware of the representation-accountability trade-off. We generalize these experimental results with observational analyses of post-electoral surveys in 26 advanced democracies. This study carries important implications, suggested in earlier formal models: poor valence candidates have an incentive to strategically increase electoral polarization to escape electoral punishment.