Imagine, Interact: Eliciting Accessible Interactions from Users with Motor Impairments via Imagined Input Devices

Paper Title

Imagine, Interact: Eliciting Accessible Interactions from Users with Motor Impairments via Imagined Input Devices

Publication Info

• Topic area: Accessible interaction design for users with motor impairments using imagined input devices.
• Keywords: Imagined input devices, motor impairments, gesture-based interaction, accessible computing, ability-based design, ability-mediating design, user elicitation, accessibility, non-physical interfaces, user-centered design.

Background and Problem

• Problem / challenge: Existing input devices (e.g., mice, remote controls, touchscreens) pose accessibility challenges for users with upper-body motor impairments due to their physicality, requiring fine motor control, gripping, or directional force. Current adaptive techniques remain constrained by the physical characteristics of these devices.
• Significance: Addressing these limitations can enable new forms of interaction, bypassing physical constraints, and improving accessibility for users with motor impairments.
• Motivation and related work: Prior research has explored adaptive interaction techniques and imaginary interfaces, but these have been conducted in isolation. The intersection of these fields—imagined input devices for accessibility—remains unexplored, leaving a gap in understanding how users with motor impairments conceptualize and operate non-physical devices.

Solution

• Proposed approach: The study investigates imagined input devices and corresponding gestures elicited from users with upper-body motor impairments, aiming to develop accessible, ability-centered interaction designs.
• Novelty:
  1. Empirical study with 11 participants eliciting imagined input devices and gestures for 20 common system functions.
  2. Identification of ten device archetypes and analysis of gesture diversity in relation to motor impairments.
  3. Design recommendations through ability-based and ability-mediating frameworks, emphasizing imagination-powered accessible computing.
• Procedure and key techniques:
  • Participants imagined input devices and gestures for 20 referents (tasks) in a smart home context.
  • Data was collected on device conceptualization, gesture types, and their relationship to motor impairments.
  • Agreement analysis was performed to assess consensus among participants.

Results

• Concrete findings:
  • 80% of imagined devices were conceptualized as embodied (e.g., hands as smartphones or remote controls).
  • Smartphones (36.4%) and remote controls (27.3%) were the most common archetypes.
  • Gestures were diverse, with 38.2% touch, 28.2% press, 24.1% mid-air, and 9.5% swipe.
  • Low consensus on gestures (.069 agreement rate) but moderate consensus on device conceptualization (.345 agreement rate).
• Advantage over baselines: Imagined devices bypass physical constraints, allowing interactions tailored to individual motor abilities and preferences.
• Experiments / evaluation:
  • Participants: 11 users with upper-body motor impairments.
  • Tasks: 20 referents across four categories (Devices, Content, Actions, Navigation).
  • Metrics: Device conceptualization, gesture types, agreement rates, and relationships to motor impairments.
• Limitations and future work:
  • Small sample size (N=11); future studies should involve larger, multicultural samples.
  • Focused on upper-body motor impairments; future work could explore co-occurring cognitive disabilities or other user groups.
  • Need for advanced gesture recognition techniques and exploration of imagined devices in broader contexts (e.g., public spaces, healthcare).

Summary

This study explores imagined input devices and gestures for users with upper-body motor impairments, revealing a strong preference for embodied devices (e.g., hands as smartphones or remote controls) and diverse, user-specific gestures. The findings highlight the potential of imagined devices to bypass physical constraints and align with individual motor abilities. Design recommendations are proposed through ability-based and ability-mediating frameworks, emphasizing the role of imagination in accessible computing. Future work should address technical challenges in gesture recognition, expand to diverse user groups, and integrate imagination into accessibility design frameworks.