Apple

apartmentCompanybar_chart#149 Global Rank

Total Papers

HCI Rank

#149

Research Areas

Years Active

Recent Publications

Mapping the Design Space of User Experience for Computer Use Agents

Large language model (LLM)-based computer use agents execute user commands by interacting with available UI elements, but little is known about how users want to interact with these agents or what design factors matter for their user experience (UX). We conducted a two-phase study to map the UX design space for comput…

IUI '26Human-LLM Collaboration +2

Athena: Intermediate Representations for Iterative Scaffolded App Generation with an LLM

It is challenging to generate the code for a complete user interface using a Large Language Model (LLM). User interfaces are complex and their implementations often consist of multiple, inter-related files that together specify the contents of each screen, the navigation flows between the screens, and the data model u…

CHI '26Visual Impairment Technologies (Screen Readers, Tactile Graphics, Braille) +3

SceneScout: Towards AI-Driven Access to Street Level Imagery for Blind Users

People who are blind or have low-vision (BLV) may hesitate to travel independently in unfamiliar environments due to uncertainty about the physical landscape. While most tools focus on in-situ navigation assistance, those supporting pre-travel assistance typically provide information about only landmarks and turn-by-t…

CHI '26Human-LLM Collaboration +2

The Way We Notice, That’s What Really Matters: Instantiating UI Components with Distinguishing Variations

Front-end developers author UI components to be broadly reusable by parameterizing visual and behavioral properties. While flexible, this makes instantiation harder, as developers must reason about numerous property values and interactions. In practice, they must explore the component’s large design space and provide…

CHI '26Human-LLM Collaboration +2

Improving User Interface Generation Models from Designer Feedback

Despite being trained on vast amounts of data, most LLMs are unable to reliably generate well-designed UIs. Designer feedback is essential to improving performance on UI generation; however, we find that existing RLHF methods based on ratings or rankings are not well-aligned with with designers' workflows and ignore t…

UIST '25Explainable AI (XAI) +2

Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors

AI policy sets boundaries on acceptable behavior for AI models, but this is challenging in the context of large language models (LLMs): how do you ensure coverage over a vast behavior space? We introduce policy maps, an approach to AI policy design inspired by the practice of physical mapmaking. Instead of aiming for…

UIST '25Human-LLM Collaboration +1

SQUIRE: Interactive UI Authoring via Slot QUery Intermediate REpresentations

Frontend developers create UI prototypes to evaluate alternatives, which is a time-consuming process of repeated iteration and refinement. Generative AI code assistants enable rapid prototyping simply by prompting through a chat interface rather than writing code. However, while this interaction gives developers flexi…

IUI '25Generative AI (Text, Image, Music, Video) +1

From Interaction to Impact: Towards Safer AI Agent Through Understanding and Evaluating Mobile UI Operation Impacts

With advances in generative AI, there is increasing work towards creating autonomous agents that can manage daily tasks by operating user interfaces (UIs). While prior research has studied the mechanics of how AI agents might navigate UIs and understand UI structure, the effects of agents and their autonomous actions—…

IUI '25Voice User Interface (VUI) Design +1

ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations

Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused understanding of images and language, but many perform poorly on UI tasks due to the lack of UI training data. In this paper, we adapt a recipe for generating paired text-image training data for VLMs to the UI domain by combining ex…

IUI '25Human-LLM Collaboration +1

VibE: A Visual Analytics Workflow for Semantic Error Analysis of CVML Models at Subgroup Level

Effective error analysis is critical for the successful development and deployment of CVML models. One approach to understanding model errors is to summarize the common characteristics of error samples. This can be particularly challenging in tasks that utilize unstructured, complex data such as images, where patterns…

View all 43 papers →

Quick Actions

Share this page

ios_share

X in mail

https://hci.top/en/institutions/Apple?source=all

At a Glance

Institution Snapshot

analytics

tag

Display Name

Apple

public

Location

United States · North America

leaderboard

Overall Rank

#149

article

Total Papers

calendar_month

Active Years

2019-2026

groups

Network

50 researchers · 0 collaborators

Research Profile

Top Research Directions

query_stats

AI & Machine Learning Interaction

Human-LLM Collaboration

Research Methods & Design Practice

Prototyping & User Testing

Data Visualization & Information Design

Interactive Data Visualization

Accessibility & Inclusive Design

Visual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)

Voice Interaction & Conversational Systems

Voice Accessibility