Understanding Changes in Partisan Strength: A Machine Learning Approach to Generate Novel Hypotheses
P13-S319-1
Presented by: Haoran Shi
Recent research emphasizes the role of partisan identity, rather than policy preferences, in shaping political behavior and polarization. While existing studies have identified various predictors of partisan strength through theory-driven approaches, less attention has been paid to bottom-up, data-driven methods for hypothesis generation. Our study employs machine learning techniques to analyze the British Election Study panel data (Waves 7-9 and 21-23) to identify novel predictors of changes in partisan strength. Using multiple classification algorithms with different feature selection and transformation methods, we find that Histogram Gradient Boosting with percentile selection consistently outperforms other approaches. Our results reveal distinct predictive patterns between moderate and extreme changes in partisan strength. While moderate changes are better predicted by political engagement metrics, extreme changes are more associated with shifts in economic attitudes and policy preferences. Notably, the models achieve better predictability in later waves (21-23) compared to earlier ones (7-9), suggesting more structured patterns of partisan change as temporal distance from major political events (e.g., Brexit) increases. These findings suggest that changes in partisan strength might follow different mechanisms depending on their magnitude, with moderate changes being more systematic and extreme changes being more environmentally driven. Our study demonstrates the value of machine learning approaches in generating novel hypotheses about partisan identity dynamics and highlights the importance of considering both individual-level attitudes and broader contextual factors in understanding partisan strength changes.
Keywords: partisan strength, machine learning, political psychology, identity, affective polarization, democratic commitment, political engagement