Comparing Design Metaphors and User-Driven Metaphors for Interaction Design

Paper Title

Comparing Design Metaphors and User-Driven Metaphors for Interaction Design

Publication Info

• Topic area: Interaction design and metaphor evaluation in user experience.
• Keywords: design metaphors, user metaphors, human-computer interaction, user experience, metaphor elicitation, ChatGPT, Twitter, YouTube, platform design, user-centered design.

Background and Problem

• Problem / challenge: Design metaphors often fail to translate effectively to users’ experiences, and there is limited understanding of how these metaphors resonate across diverse user communities.
• Significance: Understanding the alignment between design and user metaphors can improve user experience, refine design processes, and foster better communication between platforms and users.
• Motivation and related work: Prior research has explored metaphor generation and evaluation but lacks methods to systematically compare design metaphors with user-generated metaphors. This paper builds on metaphor elicitation techniques to address this gap, focusing on three platforms: ChatGPT, Twitter, and YouTube.

Solution

• Proposed approach: A comparative framework to evaluate the alignment between design metaphors (as projected by platforms) and user metaphors (elicited from users).
• Novelty:
  1. Introduced a method to surface and compare design and user metaphors for specific platforms.
  2. Developed a lightweight process for designers to collect feedback on metaphors.
  3. Explored how user metaphors reflect diverse experiences across demographic groups.
• Procedure and key techniques:
  • Surfaced 21 design metaphors by analyzing platforms’ historical web presence using the Wayback Machine.
  • Elicited 554 user metaphors through sentence-completion surveys with 300 participants across three platforms.
  • Conducted a broader survey with 926 participants to rate the reflectiveness of user metaphors.
  • Compared design and user metaphors using qualitative coding and keyword-based analysis.
  • Analyzed metaphor ratings across demographic groups using statistical methods, including ANOVA and Benjamini-Hochberg correction.

Results

• Concrete findings:
  • Only 8 out of 16 explicit keywords from design metaphors appeared in user metaphors, and most design metaphors did not directly resonate with users.
  • The highest-rated user metaphors often diverged from design metaphors, reflecting experiences like toxicity (Twitter), knowledge abundance (ChatGPT), and exploration (YouTube).
  • Significant differences in metaphor ratings were observed across racial, gender, and age groups.
• Advantage over baselines:
  • Demonstrated that user metaphors provide richer and more diverse insights into user experiences than design metaphors alone.
  • Highlighted gaps in how design metaphors communicate intended experiences to users.
• Experiments / evaluation:
  • Platforms analyzed: ChatGPT, Twitter, YouTube.
  • Methods: Qualitative coding, statistical analysis of metaphor ratings.
  • Metrics: Average ratings of metaphor reflectiveness on a 5-point Likert scale.
• Limitations and future work:
  • Limited to three platforms and a U.S.-based participant pool.
  • Relied on manual annotation for metaphor similarity.
  • Future work could explore longitudinal studies, cross-cultural contexts, and additional platforms.

Summary

This paper investigates the alignment between design metaphors and user-generated metaphors for three major platforms: ChatGPT, Twitter, and YouTube. The study finds that design metaphors often fail to resonate with users, with user metaphors reflecting a broader range of experiences, including negative and mixed sentiments. Significant demographic differences in metaphor ratings highlight the need for more inclusive design practices. The proposed methodology enables iterative refinement of metaphors and offers opportunities for longitudinal and cross-cultural studies. This work contributes to user-centered design by providing tools to evaluate and improve metaphor quality in interaction design.