A decision-theoretic representation of assistive interfaces

Paper Title

A decision-theoretic representation of assistive interfaces

Publication Info

• Topic area: Decision-theoretic modeling of assistive interfaces in Human-Computer Interaction (HCI).
• Keywords: assistive interfaces, decision theory, multi-agent systems, POSG, HCI, user modeling, adaptive systems, reinforcement learning, computational complexity, interaction modeling.

Background and Problem

• Problem / challenge: Assistive interfaces lack a unified conceptual framework, making it difficult to compare approaches, transfer insights across domains, and systematically analyze their behavior.
• Significance: A shared framework could improve understanding, collaboration, and innovation in designing assistive systems, while enabling systematic evaluation and reuse of solutions.
• Motivation and related work: Prior work spans diverse fields (AI, HCI, cognitive science) and methods (heuristics, reinforcement learning, foundation models). However, these approaches are fragmented, with no common language or structure to unify them. This paper builds on decision-theoretic models, such as MDPs, POMDPs, and POSGs, to propose a unified framework for assistive interfaces.

Solution

• Proposed approach: A decision-theoretic model of assistive interfaces, framed as a two-agent sequential decision-making problem under uncertainty.
• Novelty:
  1. Introduces a multi-agent model based on Partially Observable Stochastic Games (POSGs) tailored for HCI.
  2. Defines assistance concepts like adaptation, augmentation, delegation, and modulation within the framework.
  3. Provides a computationally implementable model, demonstrated via the CoopIHC library.
  4. Illustrates applicability through worked-out examples in pointing facilitation and artificial teaching.
• Procedure and key techniques:
  • Models assistance as interaction between a user and an assistant, both acting in a shared environment.
  • Defines key components: observation functions, internal and external transition functions, policies, and rewards.
  • Uses POSGs to capture strategic interaction, latent states, and partial observability.
  • Implements the framework in the CoopIHC library, enabling modular and reusable designs.

Results

• Concrete findings:
  • The proposed model unifies various assistive systems under a single framework, enabling systematic comparison and analysis.
  • Formal definitions of assistance types (e.g., adaptation, augmentation) are operationalized and quantifiable.
  • CoopIHC library demonstrates practical applicability, supporting implementation of diverse assistive strategies.
• Advantage over baselines:
  • Provides a richer representational power compared to single-agent models and heuristics, capturing strategic interaction and long-term effects.
  • Enables modularity and reusability of prior work, facilitating incremental advancements.
• Experiments / evaluation:
  • Two illustrative examples: pointing facilitation (BIGPoint) and artificial teaching (memory-based item selection).
  • Examples demonstrate the framework’s flexibility to represent diverse interaction patterns and strategies.
• Limitations and future work:
  • Computational complexity of solving POSGs is high; future work should focus on identifying tractable subclasses.
  • Strict assumptions (e.g., scalar rewards, static action spaces) limit applicability to open-ended tasks.
  • Definitions provided may require refinement or expansion for broader adoption.

Summary

This paper proposes a decision-theoretic model for assistive interfaces, framing assistance as a cooperative interaction between a user and an assistant under uncertainty. Using Partially Observable Stochastic Games (POSGs), the model captures strategic interaction and enables systematic definitions of assistance concepts like adaptation and augmentation. The CoopIHC library demonstrates practical implementation, supporting modular and reusable designs. While computational complexity and strict assumptions pose challenges, the framework offers a unified lens for fragmented assistive interface research, promising cross-domain transfer and incremental advancements.