Eldon Schoop

schoolApplebar_chart#2039 Global Rank

Total Papers

HCI Rank

#2039

Research Areas

Years Active

Recent Publications

Mapping the Design Space of User Experience for Computer Use Agents

Large language model (LLM)-based computer use agents execute user commands by interacting with available UI elements, but little is known about how users want to interact with these agents or what design factors matter for their user experience (UX). We conducted a two-phase study to map the UX design space for comput…

IUI '263rd AuthorHuman-LLM Collaboration +2

Athena: Intermediate Representations for Iterative Scaffolded App Generation with an LLM

It is challenging to generate the code for a complete user interface using a Large Language Model (LLM). User interfaces are complex and their implementations often consist of multiple, inter-related files that together specify the contents of each screen, the navigation flows between the screens, and the data model u…

IUI '252nd AuthorGenerative AI (Text, Image, Music, Video) +1

From Interaction to Impact: Towards Safer AI Agent Through Understanding and Evaluating Mobile UI Operation Impacts

With advances in generative AI, there is increasing work towards creating autonomous agents that can manage daily tasks by operating user interfaces (UIs). While prior research has studied the mechanics of how AI agents might navigate UIs and understand UI structure, the effects of agents and their autonomous actions—…

IUI '252nd AuthorVoice User Interface (VUI) Design +1

ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations

Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused understanding of images and language, but many perform poorly on UI tasks due to the lack of UI training data. In this paper, we adapt a recipe for generating paired text-image training data for VLMs to the UI domain by combining ex…

CHI '243rd AuthorVoice Accessibility +1

AXNav: Replaying Accessibility Tests from Natural Language

Developers and quality assurance testers often rely on manual testing to test accessibility features throughout the product lifecycle. Unfortunately, manual testing can be tedious, often has an overwhelming scope, and can be difficult to schedule amongst other development milestones. Recently, Large Language Models (L…

UIST '233rd AuthorHuman-LLM Collaboration +2

Never-ending Learning of User Interfaces

Machine learning models have been trained to predict semantic information about user interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently, most models rely on datasets that are collected and labeled by human crowd-workers, a process that is costly and surprisingly error-prone for c…

Quick Actions

Share this page

ios_share

X in mail

https://hci.top/en/authors/12084?source=all