All

Papers

Browse and search HCI research papers from All

Active Filters

Author: 8079

11 results

Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLM

Mobile apps have become indispensable for accessing and participating in various environments, especially for low-vision users. Users with visual impairments can use screen readers to read the content of each screen and understand the content that needs to be operated. Screen readers need to read the hint-text attribu…

Zhe Liu et al.Chinese Academy of Sciences

MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling

The importance of computational modeling of mobile user interfaces (UIs) is undeniable. However, these require a high-quality UI dataset. Existing datasets are often outdated, collected years ago, and are frequently noisy with mismatches in their visual representation. This presents challenges in modeling UI understan…

Sidong Feng et al.Monash University

Voicify Your UI: Towards Android App Control with Voice Commands

Nowadays, voice assistants help users complete tasks on the smartphone with voice commands, replacing traditional touchscreen interactions when such interactions are inhibited. However, the usability of those tools remains moderate due to the problems in understanding rich language variations in human commands, along…

Minh Duc Vu et al.Monash University

Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications

Mobile apps bring us many conveniences, such as online shopping and communication, but some use malicious designs called dark patterns to trick users into doing things that are not in their best interest. Many works have been done to summarize the taxonomy of these patterns and some have tried to mitigate the problems…

Jieshan Chen et al.Australian National University

AdRecommended

Learn AI Coding at CodeNow

Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work.

Explore Now

Video2Action: Reducing Human Interactions in Action Annotation of App Tutorial Videos

Tutorial videos of mobile apps have become a popular and compelling way for users to learn unfamiliar app features. To make the video accessible to the users, video creators always need to annotate the actions in the video, including what actions are performed and where to tap. However, this process can be time-consum…

Sidong Feng et al.Monash University

Guided Bug Crush: Assist Manual GUI Testing of Android Apps via Hint Moves

Mobile apps are indispensable for people’s daily life. Complementing with automated GUI testing, manual testing is the last line of defence for app quality. However, the repeated actions and easily missing of functionalities make manual testing time-consuming and inefficient. Inspired by the game candy crush with flas…

Zhe Liu et al.Chinese Academy of Sciences

Latexify Math: Mathematical Formula Markup Revision to Assist Collaborative Editing in Math Q&A Sites

Collaborative editing questions and answers plays an important role in quality control of Mathematics Stack Exchange which is a math Q&A Site. Our study of post edits in Mathematics Stack Exchange shows that there is a large number of math-related edits about latexifying formulas, revising LaTeX and converting the blu…

Suyu Ma et al.Monash University

Crowds and Collaboration

Auto-Icon: An Automated Code Generation Tool for Icon Designs Assisting in UI Development

Approximately 50% of development resources are devoted to UI development tasks [62]. Occupied a large proportion of development resources, developing icons can be a time-consuming task, because developers need to consider not only effective implementation methods but also easy-to-understand descriptions. In this study…

Sidong Feng et al.Monash University

From Lost to Found: Discover Missing UI Design Semantics through Recovering Missing Tags

Design sharing sites provide UI designers with a platform to share their works and also an opportunity to get inspiration from others' designs. To facilitate management and search of millions of UI design images, many design sharing sites adopt collaborative tagging systems by distributing the work of categorization t…

Chunyang Chen et al.Monash University

UX of AI

Gallery D.C.: Design Search and Knowledge Discovery through Auto-created GUI Component Gallery

Online communities like and GraphicBurger allow GUI designers to share their design artwork and learn from each other. These design sharing platforms are important sources for design inspiration, but our survey with GUI designers suggests additional information needs unmet by existing design sharing platforms. First,…

Chunyang Chen et al.Monash University

Expert Work

Data-Driven Proactive Policy Assurance of Post Quality in Community Q&A Sites

To ensure the post quality, Q&A sites usually develop a list of quality assurance guidelines for "dos and don'ts", and adopt the collaborative editing mechanism to fix violations of community norms. Guidelines are mostly high-level principles, and many tacit and context-sensitive aspects of the expected community norm…

Chunyang Chen et al.Monash University

Norms and Rules in Online Communities

Paper Title	Authors	Research Topics
emoji_events Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLM Mobile apps have become indispensable for accessing and participating in various environments, especially for low-vision users. Users with visual impairments can use screen readers to read the content of each screen and understand the content that needs to be operated. Screen readers need to read the hint-text attribu…	ZL Zhe Liu et al.Chinese Academy of Sciences
MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling The importance of computational modeling of mobile user interfaces (UIs) is undeniable. However, these require a high-quality UI dataset. Existing datasets are often outdated, collected years ago, and are frequently noisy with mismatches in their visual representation. This presents challenges in modeling UI understan…	SF Sidong Feng et al.Monash University
Voicify Your UI: Towards Android App Control with Voice Commands Nowadays, voice assistants help users complete tasks on the smartphone with voice commands, replacing traditional touchscreen interactions when such interactions are inhibited. However, the usability of those tools remains moderate due to the problems in understanding rich language variations in human commands, along…	MV Minh Duc Vu et al.Monash University
Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications Mobile apps bring us many conveniences, such as online shopping and communication, but some use malicious designs called dark patterns to trick users into doing things that are not in their best interest. Many works have been done to summarize the taxonomy of these patterns and some have tried to mitigate the problems…	JC Jieshan Chen et al.Australian National University
AdRecommended Learn AI Coding at CodeNow Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work. Explore Now
Video2Action: Reducing Human Interactions in Action Annotation of App Tutorial Videos Tutorial videos of mobile apps have become a popular and compelling way for users to learn unfamiliar app features. To make the video accessible to the users, video creators always need to annotate the actions in the video, including what actions are performed and where to tap. However, this process can be time-consum…	SF Sidong Feng et al.Monash University
Guided Bug Crush: Assist Manual GUI Testing of Android Apps via Hint Moves Mobile apps are indispensable for people’s daily life. Complementing with automated GUI testing, manual testing is the last line of defence for app quality. However, the repeated actions and easily missing of functionalities make manual testing time-consuming and inefficient. Inspired by the game candy crush with flas…	ZL Zhe Liu et al.Chinese Academy of Sciences
Latexify Math: Mathematical Formula Markup Revision to Assist Collaborative Editing in Math Q&A Sites Collaborative editing questions and answers plays an important role in quality control of Mathematics Stack Exchange which is a math Q&A Site. Our study of post edits in Mathematics Stack Exchange shows that there is a large number of math-related edits about latexifying formulas, revising LaTeX and converting the blu…	SM Suyu Ma et al.Monash University	Crowds and Collaboration
Auto-Icon: An Automated Code Generation Tool for Icon Designs Assisting in UI Development Approximately 50% of development resources are devoted to UI development tasks [62]. Occupied a large proportion of development resources, developing icons can be a time-consuming task, because developers need to consider not only effective implementation methods but also easy-to-understand descriptions. In this study…	SF Sidong Feng et al.Monash University
From Lost to Found: Discover Missing UI Design Semantics through Recovering Missing Tags Design sharing sites provide UI designers with a platform to share their works and also an opportunity to get inspiration from others' designs. To facilitate management and search of millions of UI design images, many design sharing sites adopt collaborative tagging systems by distributing the work of categorization t…	CC Chunyang Chen et al.Monash University	UX of AI
Gallery D.C.: Design Search and Knowledge Discovery through Auto-created GUI Component Gallery Online communities like and GraphicBurger allow GUI designers to share their design artwork and learn from each other. These design sharing platforms are important sources for design inspiration, but our survey with GUI designers suggests additional information needs unmet by existing design sharing platforms. First,…	CC Chunyang Chen et al.Monash University	Expert Work
Data-Driven Proactive Policy Assurance of Post Quality in Community Q&A Sites To ensure the post quality, Q&A sites usually develop a list of quality assurance guidelines for "dos and don'ts", and adopt the collaborative editing mechanism to fix violations of community norms. Guidelines are mostly high-level principles, and many tacit and context-sensitive aspects of the expected community norm…	CC Chunyang Chen et al.Monash University	Norms and Rules in Online Communities