15:00 - 16:40
P4-S101
Room: 1A.10
Chair/s:
Olga Gasparyan
Discussant/s:
Cantay Caliskan
Pixels, Texts and Politics: Decoding Ideological Framing on YouTube with a Multimodal Approach
P4-S101-5
Presented by: Christopher Klamm
Christopher KlammIgnatius Bernandi Jordi HarsawigunaSimone Ponzetto
University of Mannheim
Framing can have multiple dimensions, particularly when it comes to videos. While previous research has categorized YouTube channels by their political ideologies, there has been limited deep analysis of the actual content produced by these channels. Specifically, there is a limited understanding of how creators in these ideological groups frame their videos over time, and how this framing shapes the ideologies they present. Since video is the primary communication medium on YouTube, a detailed examination of video content is essential to fully comprehend the conveyed messages. To tackle these gaps, we develop a dataset of YouTube video transcripts, organized according to 14 political ideological groups (e.g. White Identitarian, Partisan Left, etc.) from different channels. We created a dataset with around 3 million video metadata entries (e.g., title and description) and selected a representative sample of around 10,000 videos for a more in-depth analysis. Using OpenAI's Whisper model, we transcribed these videos and captured screenshots to examine visual settings. We then employ TopicGPT to analyze the main and sub-frames used by different creators and ideologies over time. Additionally, we create a video description automatically derived from the frames captured in the screenshots. This multimodal approach, combining voice, image, and text analysis through Large Language Models, provides deeper insights than simpler methods that rely solely on title analysis or channel categorization. Our research offers new perspectives on how ideological influences shape YouTube content and how this can be analyzed on a large scale with multimodal models.
Keywords: ideological framing, YouTube, multimodal, large language models

Sponsors