Political scientists have drawn attention to the possibility that individuals in established democracies are affectively polarized. Our knowledge about the extent to which individuals are affectively polarized and dislike supporters of other parties is entirely based on surveys that prime respondents to think about politics and their own partisan identity and political preferences. Our study presents a design to investigate the impact of such surveys on estimated levels of affective polarization and to determine if polarization is sensitive to the context within which individuals are asked to rate parties. Using samples from the United States and Germany, we experimentally assess whether measuring affective polarization in a political versus a non-political survey produces systematically different estimates of affective polarization. Our results help clarify whether these estimates reflect how affectively polarized societies are.