Thanawit Prasongpongchai

Talk to the Hand: an LLM-powered Chatbot with Visual Pointer as Proactive Companion for On-Screen Tasks

This paper presents Pointer Assistant, a novel human-AI interaction technique for on-screen tasks. The design features a chatbot displayed as an extra mouse pointer, alongside the user's, which proactively gives feedback on user actions while directing them to relevant areas on the screen and responding to the user's…