GenFaceUI: Meta-Design of Generative Personalized Facial Expression Interfaces for Intelligent Agents

Paper Title

GenFaceUI: Meta-Design of Generative Personalized Facial Expression Interfaces for Intelligent Agents

Publication Info

• Topic area: Generative facial expression interfaces for intelligent agents
• Keywords: Generative user interfaces, facial expression interfaces, intelligent agents, meta-design, large language models, personalization, contextual expression, design tools, human–AI collaboration, expressive behavior

Background and Problem

• Problem / challenge: Existing facial expression interfaces are limited in adaptability, expressiveness, and contextual alignment. They often rely on pre-defined templates or narrowly focused machine learning models, which restrict their ability to generate dynamic, personalized, and context-sensitive expressions at run time.
• Significance: Generative facial expression interfaces can enhance intelligent agents' usability, engagement, and trustworthiness by enabling dynamic and personalized interactions. However, achieving control, coherence, and alignment with design intent in these interfaces remains a challenge.
• Motivation and related work: Prior work has explored static templates, machine learning models, and LLM-driven expression generation, but these approaches lack comprehensive frameworks for run-time generation and tools that support designers in shaping generative spaces. This paper builds on the emerging paradigm of generative user interfaces and addresses the gap by introducing a meta-design framework.

Solution

• Proposed approach: The Generative Personalized Facial Expression Interface (GPFEI) framework and its implementation in the GenFaceUI tool.
• Novelty:
  1. Introduction of the GPFEI framework, which integrates rule-bounded design spaces, character identity, and context-driven variation.
  2. Development of GenFaceUI, a proof-of-concept meta-design tool for creating and testing generative facial expression interfaces.
  3. Empirical insights from a qualitative study with designers, highlighting challenges and opportunities in meta-design practices.
• Procedure and key techniques:
  • The GPFEI framework structures facial expression generation into four components: personalized face generation, contextual facial expression, context mapping, and meta-design.
  • GenFaceUI operationalizes the framework with two modes (Design and Test) and five components (Face Template Canvas, Semantic Tag Editor, Design Rule Editor, Context Mapping Rule Editor, Runtime Expression Simulator).
  • Designers author templates, define rules, and simulate outputs iteratively, using structured prompts and visual editing tools.

Results

• Concrete findings:
  • Participants generated 268 test faces, authored 123 rules, and created 236 interface elements across three design tasks.
  • Designers perceived gains in visual consistency and creative support but noted limitations in expressiveness, predictability, and fine-grained control.
• Advantage over baselines:
  • Compared to traditional template-based or parameter-driven approaches, GenFaceUI allowed for greater personalization, contextual adaptation, and rule-based control of generative outputs.
• Experiments / evaluation:
  • A qualitative study with 12 designers explored their experiences with GenFaceUI across three tasks: designing a chatbot face, customizing a service robot, and creating a virtual AI companion.
  • Data sources included system logs, interviews, and think-aloud reflections, analyzed through thematic coding.
• Limitations and future work:
  • Limitations include lack of temporal continuity, multimodal coordination, and fine-grained professional controls.
  • Future work will focus on enhancing expressive capabilities, improving professional design support, and evolving GenFaceUI into a collaborative environment.

Summary

This paper introduces the GPFEI framework and GenFaceUI tool to address challenges in generative facial expression interfaces for intelligent agents. The framework structures design-time specifications and run-time variations, while GenFaceUI enables designers to author templates, define rules, and test outputs iteratively. A qualitative study with 12 designers demonstrated gains in controllability and creative support but highlighted needs for richer expressiveness, predictability, and professional-level controls. This work advances the understanding of meta-design in generative interfaces and outlines pathways for future research and application in intelligent agents.