The relationship between voters’ ideology and parties’ ideological perception has been studied through linear projection effects. In this paper, I propose a complementary framework in which the differences between extreme and moderate voters are analyzed through a multilevel strategy. By combining cross-sectional European electoral data and individual panel data from Germany and Spain, I show that ideologically extreme voters perceive the parties as holding more extreme ideological positions. This happens regardless of whether voters and parties are leftist or rightist. Cross-sectional data shows the divergence between extreme voters from various European countries, which accounts for differences of polarization among the countries and variance between European parties. Individual data shows that polarization has a double component: extreme voters are farther away from their non-supported parties not only because of voters’ extreme positions but also because they are allocated more to the extremes by these voters. Therefore, I show voters on different sides of the spectrum more and more withdraw into their own realities, making democratic compromise difficult on this qualitatively distinct level. In all, my results suggest that projection models of voters’ perception and polarization studies should consider voters’ extremism as one of the variables explaining parties ’perceptions.