Ada Yi Zhao

Guided Reality: Generating Visually-Enriched AR Task Guidance with LLMs and Vision Models

Large language models (LLMs) have enabled the automatic generation of step-by-step augmented reality (AR) instructions for a wide range of physical tasks. However, existing LLM-based AR guidance often lacks rich visual augmentations to effectively embed instructions into spatial context for a better user understanding…