AiModerator: A Co-Pilot for Hyper-Contextualization in Political Debate Video

DOI

Political debates are essential in political discourse for democratic societies. Advancements in technology have significantly transformed the structure of political debates, the ways in which politicians communicate, and the platforms through which audiences engage with them. Originally a forum for improving understanding, political debates have increasingly favored theatrics over substance, risking young adult disengagement. To bring substance back to this medium we developed AiModerator, a political debate co-pilot acting as a Multimodal Conversational Agent (MCA). AiModerator aims to promote engagement while improving understanding by analyzing video content to provide contextually relevant information. This consolidated information facilitates understanding while keeping users synchronized with the debate viewing experience. Our system builds upon multimodal techniques, integrating computer vision and large language models to demonstrate ways of improving content delivery and engagement. AiModerator's backend system extracts events from identified speech data, allowing the user to interact with these events through a touch interface on an iPad application. We address three key topics: evaluating young adults' engagement, satisfaction, and preference compared to traditional second screening, and determining whether AiModerator can improve subjective understanding. To evaluate these measures we conducted a mixed-method evaluation (n=20) within-group design A-B study. Our analysis found AiModerator excelled in promoting engagement and satisfaction while delivering clear, contextually relevant information to the user which improved their understanding of debate topics more than the second screening mode. Our qualitative analysis offers broader insights, particularly in terms of a trade-off between automation and information consolidation versus autonomy and control.