TUI as Social Entity: a Study of Joint-actuation and Turn-taking-actuation in Actuated-interfaces

Title of the Paper

TUI as Social Entity: a Study of Joint-actuation and Turn-taking-actuation in Actuated-interfaces

Paper Information

• Subject Area: Human-Computer Interaction (HCI), Tangible User Interfaces (TUI), Autonomous Behavior Interfaces
• Keywords: TUI, Autonomous Behavior Interfaces, Self-actuation, Automatic Behavior, Social Interaction

Research Background and Problem

• Research Problem:
  The authors focus on extending the autonomy of Tangible User Interfaces (TUI), specifically on designing an interactive system that can function both as an interface and as an independent social entity. The study particularly explores two interaction modes: asynchronous "Turn-taking-actuation" and synchronous "Joint-actuation."

• Core Challenges:
  Existing TUI research rarely addresses whether the system can be perceived by users as a social entity with independent intentions. In particular, changes in users' sense of "control" and their emotional responses during interaction require further evaluation.

• Significance of the Research:
  Enhancing TUIs with autonomous behavior capabilities may transform users' social experiences and perceptions of autonomous systems, which is crucial for the future development of natural human-machine collaboration systems.

• Related Work Review:

  1. Existing TUI studies, such as [11][14][36], focus on input/output coupling but do not explore the social implications of deep autonomy.
  2. The application of autonomous functionality in shape-changing interfaces [8][39] and self-actuating interfaces [27][28][30] has been explored, but there is a lack of comparative analysis on the social and emotional impacts of different interaction modes.
  3. While the field of Human-Robot Interaction (HRI) has studied social interactions with non-humanoid robots, such research rarely involves direct physical manipulation and does not focus on shared control between users and systems.

Solution

• Method Overview:
  A system prototype combining physical and autonomous behavior was proposed. This system functions both as a tangible interactive interface and as a social entity with independent behavior. Experiments were conducted to evaluate two interaction modes (Turn-taking-actuation and Joint-actuation).

• Innovations:

  1. Extending TUIs from simple tools to autonomous social entities, enabling both technical operations and social interactions.
  2. Proposing and implementing two interaction control modes, "Joint" and "Turn-taking," and analyzing their social attributes from the user's perspective.

• Key Technologies:

  • Hardware: Raspberry Pi and Dynamixel servo motors for four-pillar mechanical control, integrated with the "Butter" robot prototype platform.
  • Software: Modules for letter recognition, word matching, and automatic letter completion developed using Python and JavaScript.
  • Experimental Tools: The Wizard-of-Oz (WoZ) method was used to simulate partial automated responses during the interaction process.

• Implementation Steps:

  1. Develop and refine the hardware and software platform to implement the letter-rotating pillar system and word-matching functionality.
  2. Design two interaction modes:
     • Turn-taking-actuation: Users and the system alternately complete a word-building task.
     • Joint-actuation: Users and the system operate simultaneously, with the system completing the word-building task when appropriate.
  3. Conduct experiments with participants (including interviews) to collect behavioral response data.

Research Findings

• Specific Findings:

  1. All 23 participants perceived the system as a social interaction entity.
  2. The "Turn-taking-actuation" mode was primarily interpreted as competitive behavior, while the "Joint-actuation" mode was mainly interpreted as collaborative behavior.
  3. In the "Joint-actuation" mode, users exhibited stronger emotional responses, including both positive (e.g., a sense of support) and negative (e.g., a sense of loss of control) reactions.

• Experimental Comparison Conclusions:

  • Turn-taking-actuation:
    • Social Experience: Mostly interpreted as "competition."
    • Sense of Control: Users felt they had full control over the interaction.
  • Joint-actuation:
    • Social Experience: Tended towards "collaboration," but users also exhibited high sensitivity to control and emotional fluctuations.
    • Sense of Control: Users experienced emotions of "struggling for control" as well as moments of "relinquishing control" in certain scenarios.

• Advantages:

  1. Compared to traditional TUIs, this study introduces a social design dimension, enhancing users' sense of engagement and emotional responses to the interface.
  2. The two modes respectively enhance emotional experiences in different interaction scenarios: competition or collaboration.

• Limitations:

  1. The experimental content was limited to a word-building game, which may not fully generalize to other complex task environments.
  2. The use of the WoZ method in the Turn-taking-actuation mode may have influenced participants' perception of the system's autonomy.
  3. Qualitative interviews may be subject to researchers' subjective biases.

• Future Directions:

  1. Investigate adaptive control technologies (e.g., context awareness and real-time responses) to further optimize the user experience in the "Joint-actuation" mode.
  2. Expand experimental task scenarios to more practical applications (e.g., educational and medical assistive tools).
  3. Explore the perception of social entities and collaboration models in multi-user interaction scenarios.