Does intra-district ideological heterogeneity cause party polarization in Congress? Instead of analyzing elite polarization and mass polarization separately, this study investigates the role of representational linkages between legislators and voters at the congressional district level. Specically, I examine how ideological distributions of district-level voters' preferences (the median voter's ideology and ideological heterogeneity) affect congressional candidates' policy position strategies during the election by using the Cooperative Congressional Election Study (CCES). To demonstrate this, several scaling methods are used in combination. First, I utilize Bayesian Aldrich-McKelvey (BAM) Scaling in order to recover common ideological space and to estimate the ideological positions of all congressional candidates from voters' perceptual data, which are comparable across districts. Second, item response theory (IRT) analysis on policy issues enables the recovery of constituencies' latent ideological dimensions. Third, representative values on district voters' ideological distributions are obtained in order to identify the characteristics of the geographic distribution of constituency preferences. Empirical results reveal that district ideological heterogeneity interacted with district median ideology, leading to ideologically extreme positioning and suggesting that the uncertainty regarding the voters' ideological preferences heightens polarization.