Classifying negative campaigning at scale: A study of candidates' social media communication across eight German elections
P2-1
Presented by: Marius Saeltzer
The importance of attack strategies, i.e., negative campaigning (NC), has grown considerably in recent years - especially on social media (SM). Reliably identifying NC at large scale in SM is a challenge to researchers. Most literature focuses on the US, where the two-party system makes NC easier to identify then in multiparty systems with multiple and potentially dynamic targets. Here, we also find coalition signals and acclaims of other parties.
To tackle this, we apply a detailed strategy-level manual coding (attack, acclaim) to N∼50,000 posts (Facebook, Instagram, Twitter) from candidates standing in three federal and five state elections in Germany from 2013 to 2021. Since NC is relatively rare and often combined with other strategies, post level coding seems insufficient. We introduce a novel approach of annotator rationales to overcome previous limitations. In addition to coding the strategy, human coders explicitly mark parts of text indicative for their coding decision. We create a more precise training set for supervised learning and use a BERT transformer to maximize performance, comparing it to numerous algorithms.
Results show that NC is relatively rare (< 6 percent of all messages). We correctly identify about 60% of all attacks out of sample with an overall accuracy of .94 on unbalanced data. Using annotator rationales improves the F1 score of attacks compared to a post-level coding from .60 to .67. While NC is identifiable via automated text analysis, social media communication makes it difficult to detect specific campaign strategies, underscoring the fruitfulness of our targeted approach.
To tackle this, we apply a detailed strategy-level manual coding (attack, acclaim) to N∼50,000 posts (Facebook, Instagram, Twitter) from candidates standing in three federal and five state elections in Germany from 2013 to 2021. Since NC is relatively rare and often combined with other strategies, post level coding seems insufficient. We introduce a novel approach of annotator rationales to overcome previous limitations. In addition to coding the strategy, human coders explicitly mark parts of text indicative for their coding decision. We create a more precise training set for supervised learning and use a BERT transformer to maximize performance, comparing it to numerous algorithms.
Results show that NC is relatively rare (< 6 percent of all messages). We correctly identify about 60% of all attacks out of sample with an overall accuracy of .94 on unbalanced data. Using annotator rationales improves the F1 score of attacks compared to a post-level coding from .60 to .67. While NC is identifiable via automated text analysis, social media communication makes it difficult to detect specific campaign strategies, underscoring the fruitfulness of our targeted approach.