Yaxian Shan

SketchGPT: A Sketch-based Multimodal Interface for Application-Agnostic LLM Interaction

Human interaction with large language models (LLMs) is typically confined to text or image interfaces. Sketches offer a powerful medium for articulating creative ideas and user intentions, yet their potential remains underexplored. We propose SketchGPT, a novel interaction paradigm that integrates sketch and speech in…