Midair Balloon Interface: A Soft and Lightweight Midair Object for Proximate Interactions

Title of the Paper

Midair Balloon Interface: A Soft and Lightweight Midair Object for Proximate Interactions

Paper Information

• Research Field: Human-Computer Interaction, 3D User Interfaces (3D UIs)
• Keywords: Midair UIs, Soft interfaces, Tangible interfaces, Ultrasound, Interaction devices, Mixed/Augmented reality

Research Background and Problem

• Identified Problems or Challenges:

  • Current 3D user interfaces (3DUIs) struggle to balance speed and safety, particularly in scenarios involving direct touch interactions.
  • Traditional flying devices (e.g., drones) used for midair interactions pose safety risks due to their hard components.
  • Existing ultrasonic levitation technologies typically focus on controlling small particles or large spheres, which do not support direct touch; physical contact disrupts the acoustic field.

• Significance:

  • Providing a safe, fast, and easy-to-operate midair object interaction interface is crucial for augmented reality, virtual reality, and other human-computer interaction technologies.
  • Improving the safe manipulation of midair objects will enhance various application scenarios, such as education, entertainment, and industrial operations.

• Research Motivation:

  • To develop soft and safe midair objects (e.g., helium balloons) that support direct manipulation while maintaining high flexibility and speed.
  • To leverage ultrasonic non-contact driving technology to design a lightweight, soft midair interaction interface that overcomes the operational inconvenience and low safety of current technologies.

Solution

• Proposed Approach:

  • Use non-contact driving based on Airborne Ultrasound Phased Arrays (AUPAs) to manipulate helium-filled balloons.
  • Introduce a time-sliced driving algorithm to optimize multi-directional balloon control.
  • Implement touch- and vision-based interaction techniques and achieve vibrational feedback on the balloon surface through amplitude-modulated ultrasound.

• Innovations:

  • Developed a midair user interface based on soft and lightweight helium balloons, offering high safety even at high speeds without causing harm to users.
  • Extended traditional ultrasonic-driven particle manipulation to 3D control of midair balloons, supporting direct touch interactions.
  • Proposed a novel ultrasound force feedback method that focuses vibrational feedback on the object held by the user rather than directly on the skin.

• Implementation Steps and Techniques:

  1. Use high-frequency stereo cameras and depth sensors to track the balloon's position and user gestures.
  2. Configure AUPAs to generate acoustic focal points in designated spaces, controlling the balloon's trajectory.
  3. Apply a PID control algorithm to adjust the phase and amplitude of ultrasonic waves in real time, ensuring stable and precise balloon position control.
  4. Provide vibrational feedback to users through amplitude-modulated ultrasound.

Research Outcomes

• Specific Results:

  • The prototype system achieved real-time 3D control of a 10 cm diameter balloon, with significant speed improvements: up to 890 mm/s horizontally and 698 mm/s vertically.
  • Supported dynamic tracking based on visual input and touch-based operation.
  • Enabled tactile feedback through ultrasonic vibrations, allowing users to perceive up to 11 distinct intensity levels of vibration.
  • Demonstrated the practicality of this technology in various experiments and user interaction scenarios.

• Advantages over Existing Solutions:

  • Safety: The balloon is made of soft and lightweight materials, completely eliminating the risk of injury from collisions.
  • Flexibility: Achieved efficient non-contact 3D manipulation, supporting complex curved trajectories and multiple interaction methods.
  • Convenience: Provided unique tactile feedback to enhance user experience, a feature not covered by traditional midair UI technologies (e.g., drones).

• Experimental and Evaluation Results:

  • Experimental measurements showed that the system's maximum error range in multiple directions was less than 22 mm, demonstrating high precision in control.
  • User experiments validated the effectiveness of ultrasonic vibrational feedback and the adjustable range of perceived intensity.

• Limitations and Future Directions:

  • Limitations:
    • The current algorithm requires further optimization to improve the efficiency of acoustic force generation.
    • The system is currently limited to helium balloons and does not yet support other media.
    • The system needs enhancements to handle larger workspace environments.
  • Future Directions:
    • Improve algorithms and hardware to increase driving force and expand the range of controllable objects (e.g., non-buoyant midair objects).
    • Extend the system's workspace and enhance safety features, such as adding auditory protection.
    • Explore more application scenarios, such as educational demonstrations, medical assistance, and augmented reality interactions.

Conclusion

This paper proposes a midair balloon user interface based on non-contact ultrasound technology. The system achieves safe and high-speed midair manipulation and demonstrates a range of potential application scenarios through the innovative use of helium balloons. The research also shows significant potential for future development in algorithm optimization, expanding to other object types, and enhancing user interaction diversity.