SurfAirs: Surface + Mid-air Input for Large Vertical Displays

Document Title

SurfAirs: Surface + Mid-air Input for Large Vertical Displays

Document Information

• Domain: Human-Computer Interaction, specifically input techniques for large vertical screen environments
• Keywords: Tangible Input, Wall Displays, Mid-air Input, Surface Input, Hybrid Interaction

Research Background and Problem

• Identified Issues or Challenges:

  • Large vertical displays provide high-resolution, expansive workspaces, allowing users to physically navigate for close-up detail observation or distant wide-angle views. However, designing input techniques that span such large interaction spaces is highly challenging.
  • Existing gesture recognition methods (e.g., bare-hand input) are intuitive but suffer from issues such as recognition inaccuracies, lack of stability, and conflicts with users' regular movements.
  • Handheld controllers can provide indirect control but require users to continuously hold the device, which interferes with direct touch interactions on the screen.

• Importance:

  • With the widespread adoption of large displays, particularly in collaborative and data analysis environments, input techniques that efficiently and seamlessly support both close-range touch and distant mid-air operations are crucial for enhancing user experience and functionality.

• Research Motivation and Related Work:

  • Recent studies on physical controllers for touch desktops and wall displays have demonstrated higher efficiency and comfort. However, these controllers either support screen operations exclusively or mid-air operations exclusively, without integrating the two.
  • The authors aim to explore a hybrid input design that supports both screen and mid-air operations and enables seamless transitions between the two.

Solution

• Proposed Solution:

  • The authors introduce SurfAirs, a physical controller capable of operations on screen surfaces (Surface) and in mid-air (Mid-air), allowing transitions between the two modes within a single interaction.

• Innovations:

  • SurfAirs is a truly hybrid physical controller, enabling continuous operations from surface to mid-air, addressing the limitations of existing input technologies.
  • It combines tactile feedback and mechanical switches to support precise selection and multidimensional control, with both handheld and fixed-use functionalities.

• Implementation Steps and Key Technologies:

  • SurfAir Design and Fabrication:
    • Rapid prototyping through modular manufacturing processes, including suction cup mechanisms and handles with rolling switches.
    • Two grip designs were prototyped: Torch shape and Handle shape.
  • Input States and Transitions:
    • Detailed definitions of touch input and mid-air input states and transitions, including drag, rotate, and zoom dimensions.
    • Seamless transitions supported, such as moving from screen operations to mid-air operations and vice versa.
  • Tracking and Recognition:
    • Utilizing a multi-touch framework and infrared tracking system to simultaneously monitor the controller's position and switch states, ensuring precision and stability.

Research Outcomes

• Specific Results:

  • SurfAirs performed exceptionally well in experiments, significantly outperforming bare-hand input. Users found operations like pointing and docking more stable, precise, and comfortable with SurfAirs.
  • SurfAirs eliminates the dependency on traditional gesture input while reducing the burden of requiring users to wear or hold devices.

• Advantages:

  • Smooth transitions between mid-air and surface operations, providing users with a more stable and confident interaction experience.
  • Superior performance in controlling multiple degrees of freedom, precise selection, and complex operational tasks compared to bare-hand input.

• Experiment or Evaluation Results:

  • The first experiment compared SurfAirs with bare-hand input in pure mid-air operation tasks:
    • SurfAirs demonstrated shorter task completion times, lower error rates, and higher subjective user confidence in interaction.
    • The Handle shape outperformed the Torch shape.
  • The second experiment evaluated hybrid tasks involving both surface and mid-air operations:
    • SurfAirs significantly improved accuracy and stability, particularly in screen-based drag, rotate, and zoom operations.
    • During transitions, the Grab transition technique slightly outperformed the Flick technique.

• Limitations and Future Directions:

  • The experimental environment was a specific lab setup involving wall displays and optical tracking systems. Future work should validate SurfAirs' applicability in other practical environments (e.g., non-optical screens or capacitive screens).
  • Current research focused on single-user and basic task scenarios; future studies could expand to multi-user, multi-controller, and high-cognitive-demand task scenarios.
  • Explore applications of SurfAirs in complex multi-display environments, such as integrating vertical displays with horizontal touch desktops.