LLooM: Weaving Stories and Probing Experiences of Language Technologies

Paper Title

LLooM: Weaving Stories and Probing Experiences of Language Technologies

Publication Info

• Topic area: Human-Computer Interaction (HCI) and probe design for studying experiences with language technologies.
• Keywords: LLooM, feminist HCI, probe design, storytelling, language technologies, NLP, collective sensemaking, alignment, valence, participatory design.

Background and Problem

• Problem / challenge: Existing methods for evaluating language technologies often reduce user experiences to technical metrics or singular notions of success, failing to capture the situated, plural, and contradictory nature of interactions.
• Significance: Language technologies like voice assistants, chatbots, and large language models mediate communication and influence social meaning. Understanding their impact requires methods that account for diverse and complex user experiences.
• Motivation and related work: Prior work on probes, storytelling, and feminist HCI has explored subjective and critical approaches to user research. However, these methods often lack mechanisms for participants to challenge or reshape the research framework itself. This paper builds on these traditions to address this gap.

Solution

• Proposed approach: LLooM, a facilitated and collective probe using storytelling and textile weaving to capture user experiences with language technologies.
• Novelty:
  1. Introduces methodological negotiation, enabling participants to critique and reshape the research framework.
  2. Supports collective sensemaking through the visible accumulation of participant contributions.
  3. Operationalizes feminist HCI principles, including pluralism, participation, and self-disclosure, in probe design.
  4. Combines material crafting and storytelling to elicit situated and reflective accounts.
• Procedure and key techniques:
  1. Participants reflect on an interaction with a language technology.
  2. They write a short story on a fabric strip, choosing colors to indicate alignment (blue for misaligned, yellow for aligned).
  3. They weave the strip into one of two looms, representing positive or negative valence.
  4. Facilitated dialogue and the visible accumulation of contributions enable reflection and collective interpretation.

Results

• Concrete findings:
  • 56 participants engaged with LLooM during a six-hour deployment.
  • Participants used facilitated dialogue to reinterpret experiences, combine categories, or critique the alignment-valence framework.
  • Collective contributions revealed patterns, contradictions, and shared concerns, such as contrasting experiences with autocorrect across linguistic contexts.
  • Material choices supported selective disclosure, allowing participants to control the visibility of their contributions.
• Advantage over baselines: LLooM extends traditional probes by enabling participants to influence the research methodology and fostering collective sensemaking through material accumulation.
• Experiments / evaluation:
  • Deployment at a public science festival with diverse participants.
  • Data collected through participant contributions, field notes, and reflexive thematic analysis.
  • Facilitated interactions and visible contributions supported methodological negotiation and collective interpretation.
• Limitations and future work:
  • Limited depth of engagement due to the short-term, public setting.
  • Participants could not introduce new analytical categories beyond the provided alignment-valence scaffold.
  • The elite institutional context may have excluded marginalized groups.
  • Future work should explore community-led contexts, longer deployments, and mechanisms for participants to define new evaluative dimensions.

Summary

LLooM is a novel probe designed to capture the situated and contradictory experiences of language technologies through storytelling and textile weaving. It enables participants to negotiate the research framework and engage in collective sensemaking, operationalizing feminist HCI principles such as pluralism and self-disclosure. Deployed with 56 participants at a science festival, LLooM revealed how material and facilitated interactions can surface methodological critiques and shared patterns of experience. Future iterations should explore more inclusive contexts and allow participants to introduce new analytical categories.