Gang Li

schoolGooglebar_chart#1208 Global Rank

Total Papers

HCI Rank

#1208

Research Areas

Years Active

Recent Publications

UICrit: Enhancing Automated Design Evaluation with a UI Critique Dataset

Automated UI evaluation can be beneficial for the design process; for example, to compare different UI designs, or conduct automated heuristic evaluation. LLM-based UI evaluation, in particular, holds the promise of generalizability to a wide variety of UI types and evaluation tasks. However, current LLM-based techniq…

CHI '242nd AuthorHuman-LLM Collaboration +1

Automatic Macro Mining from Interaction Traces at Scale

Macros are building block tasks of our everyday smartphone activity (e.g., "login", or "booking a flight"). Effectively extracting macros is important for understanding mobile interaction and enabling task automation. These macros are however difficult to extract at scale as they can be comprised of multiple steps yet…

CHI '232nd AuthorVoice User Interface (VUI) Design +1

Enabling Conversational Interaction with Mobile UI using Large Language Models

Conversational agents show the promise to allow users to interact with mobile devices using language. However, to perform diverse UI tasks with natural language, developers typically need to create separate datasets and models for each specific task, which is expensive and effort-consuming. Recently, pre-trained large…

CHI '223rd AuthorExplainable AI (XAI) +1

Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis

UI designers often correct false affordances and improve the discoverability of features when users have trouble determining if elements are tappable. We contribute a novel system that models the perceived tappability of mobile UI elements with a vision-based deep neural network and helps provide design insights with…

CHI '221st AuthorPrototyping & User Testing +1

Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale

The layout of a mobile screen is a critical data source for UI design research and semantic understanding of the screen. However, UI layouts in existing datasets are often noisy, have mismatches with their visual representation, or consists of generic or app-specific types that are difficult to analyze and model. In t…

UIST '212nd AuthorVoice User Interface (VUI) Design +1

Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning

Mobile User Interface Summarization generates succinct language descriptions of mobile screens for conveying important contents and functionalities of the screen, which can be useful for many language-based application scenarios. We present Screen2Words, a novel screen summarization approach that automatically encapsu…

UIST '212nd AuthorInteractive Data Visualization +1

HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions

We present HelpViz, a tool for generating contextual visual mobile tutorials from text-based instructions that are abundant on the web. HelpViz transforms text instructions to graphical tutorials in batch, by extracting a sequence of actions from each text instruction through an instruction parsing model, and executin…

CHI '212nd AuthorCrowdsourcing Task Design & Quality Control +1

Spacewalker: Rapid UI Design Exploration Using Lightweight Markup Enhancement and Crowd Genetic Programming

User interface design is a complex task that involves designers examining a wide range of options. We present Spacewalker, a tool that allows designers to rapidly search a large design space for an optimal web UI with integrated support. Designers first annotate each attribute they want to explore in a typical HTML pa…

Quick Actions

Share this page

ios_share

X in mail

https://hci.top/en/authors/6431?source=all