Expressivity in Interaction, a Framework for Design

Document Title

Expressivity in Interaction: a Framework for Design

Document Information

• Subject Area: Expressive design in Human-Computer Interaction (HCI)
• Keywords: Expressive interaction, research-driven design, framework, design case studies, multimodal interaction, shape-changing interfaces, materiality

Research Background and Issues

• Problems or Challenges:

  • The authors observe that "expressivity" is frequently discussed in the HCI field but lacks a unified definition.
  • Expressivity encompasses multiple perspectives, such as human emotional expression, artifact expression, and the combination within human-computer interaction.
  • Current research reveals a significant gap between theory and practice, making it difficult to provide clear operational guidance for designers.

• Significance:

  • With the advancement of artificial intelligence, shape-changing interfaces, and material technologies, interactive technologies are increasingly integrated into daily life, occupying an essential socio-cultural position.
  • Designing tools that offer greater expressivity for multi-party participants and physical interaction can enhance user experience and foster emotionally rich human-computer interactions.

• Research Motivation and Related Work:

  • By conducting a meta-analysis of existing research, the study leverages "Research through Design" to demonstrate how theory and tools can support expressive design.
  • The research aims to address the lack of clear definitions and frameworks regarding "expressivity" in HCI studies.

Solution

• Main Methods and Framework:

  • Multi-perspective Framework:
    1. Human Expressivity: Explores the freedom of expressing bodily movements, emotions, and intentions.
    2. Artefact Expressivity: Investigates the expression of systems through form, dynamic behavior, and visual elements.
    3. Interaction Expressivity: Focuses on the interaction between human and system expressions.
  • Proposes the "Framework for Expressivity in Interaction," which includes key elements influencing design:
    • Emotional State
    • Action-Perception Loops
    • Multimodality
    • Temporal Form
    • Subtlety and Ambiguity
    • Skill Development
    • Socio-Cultural Context

• Research Methods:

  1. Literature Review: Analyzes definitions, case studies, and theoretical frameworks related to expressivity in HCI.
  2. Design Exemplars: Selects 29 papers from ACM and IEEE domains describing clear design cases, extracting nine categories of expressivity through thematic analysis.
  3. Case Validation: Demonstrates the framework's practical application through case studies (e.g., the shape-changing soft robotic interface FRANK).

• Innovations:

  • Integrates human, artifact, and interaction perspectives into a unified framework, addressing current academic debates in HCI.
  • Transforms research-driven design into a theoretical construction tool.
  • Provides actionable design principles to bridge the gap between design theory and practical processes.

Research Outcomes

• Specific Findings:

  • The quality of expressivity is influenced by factors such as input and output diversity, coherence of interaction loops, handling of ambiguity in experiences, and the potential for long-term skill development.
  • Dynamic morphological changes (e.g., shape-changing interfaces) are a critical component of expressive interaction.

• Advantages Compared to Existing Solutions:

  • Considers the holistic perspective of human-artifact-interaction relationships.
  • Highlights the core role of multimodality and ambiguity in design.
  • Offers clear elements to guide the design of expressive interactions.

• Experiments and Evaluation Results:

  • The framework was applied in the design of the shape-changing interaction system (FRANK):
    • FRANK's design integrates dynamic material morphology with user perception, supporting nuanced hand motion interactions and embodying expressivity in human-computer interaction.
    • The case validated the applicability of framework elements such as "Action-Perception Loops," "Temporal Form," and "Subtlety."

• Limitations and Future Directions:

  1. Limitations:
     • The framework is currently focused on the conceptual validation phase of design and has not been tested in large-scale complex scenarios.
     • It does not fully address multi-user and multi-artifact interactions, such as those in IoT or robotic swarm environments.
  2. Future Directions:
     • Extend the framework to domains requiring skill development, such as music creation.
     • Explore socio-cultural dimensions in complex scenarios.
     • Redefine the design framework for multi-agent expressive ecosystems by integrating emerging technologies (e.g., IoT, robotics).