WatchWithMe: LLM-Based Interactive Guided Watching of Review Videos

DOI

Videos are a popular way for viewers to follow topics of interest. In areas such as product and technology reviews, videos often present in-depth perspectives in a compact fashion, driving viewers to look for additional explanations. We propose WatchWithMe, an automatic approach that provides viewers in-context guided watching during video playback. Powered by large language models, WatchWithMe generates guided materials from the video transcript as if creating a reading guide, including summaries, highlights, and question prompts. WatchWithMe reveals relevant information responsive to the spoken content in a review video. Viewers skim and prompt in our text-based conversational UI, to which we automatically expand the video viewing context to the model for contextual responses. We evaluated WatchWithMe with public videos and collected feedback from 20 participants. Findings showed that our method encouraged viewers to seek out viewpoints or confirmations related to the video topics.