LipIO: Enabling Lips as both Input and Output Surface

Title of the Paper

LipIO: Enabling Lips as both Input and Output Surface

Paper Information

• Research Area: Human-Computer Interaction, specifically tactile-based input-output interfaces
• Keywords: Haptics, lips, tongue, on-skin interface, eyes-free interaction, hands-free interaction, electrotactile stimulation, capacitive touch, accessibility, tactile feedback

Research Background and Issues

• What problems or challenges did the authors identify?

  • Traditional human-computer interaction methods (e.g., finger touch and voice interaction) are limited in scenarios where users' eyes and hands are occupied.
  • Existing oral interfaces (e.g., tongue-based input devices) typically only support input or output, making it difficult to achieve combined input and output functionality.
  • Current oral devices are often bulky, rely on wired connections, and are unsuitable for everyday use.

• Why is this issue important?

  • The inconvenience of using eyes and hands for interaction affects users' task efficiency and experience, particularly in complex environments or multitasking scenarios.
  • New interaction methods are crucial for improving accessibility, such as supporting users with mobility impairments.

• Research Motivation and Related Work

  • Previous studies have focused on input-output devices for eyes or hands, brain interfaces, and oral-based input methods, but there is limited research on "simultaneous input and output."
  • The high sensitivity of lips and the flexibility of the tongue provide inspiration for interface design.
  • This study aims to develop a compact, lip-adhering device to address the limitations of existing technologies.

Solution

• What methods or solutions did the authors propose?

  • Developed a novel device called "LipIO," which uses lips as a surface for both input and output, supporting electrotactile output and capacitive touch-based input.
  • The LipIO device consists of two layers of flexible electrodes: one facing outward for capacitive touch input and the other facing the lips for electrotactile stimulation output.
  • Provides real-time tactile feedback, allowing users to feel touch-related tactile stimulation when their tongue interacts with their lips.

• What is innovative about this solution?

  • Combines input and output functionalities on the same surface of the user's lips, a first in this field.
  • Achieves a transparent interaction experience similar to finger touch devices through tactile feedback mechanisms.
  • Enables efficient interaction even when users' eyes and hands are occupied.

• What are the implementation steps and key technologies used?

  1. Device Design:
     • Two-layer electrode stack design (for input and output), combined with a plastic isolation layer and flexible adhesive layer.
     • Electrode arrays manufactured using conductive tape and inkjet printing technology.
  2. Signal Processing and Feedback:
     • Capacitive touch sensor (MPR121) used for input detection.
     • Time multiplexing technology employed for multi-point tactile feedback.
  3. User Studies:
     • Designed and validated three experiments to optimize the device's interaction performance, including perception accuracy, the role of tactile feedback, and optimizing the number of electrode points.
  4. Application Implementation:
     • Developed prototype applications for various scenarios, such as door lock controllers, electric bike navigation, and lip-based games.

Research Outcomes

• What specific outcomes were achieved?

  • Users could efficiently perform simultaneous input and output through their lips, with an average accuracy of 93%.
  • Developed prototypes for multiple application scenarios, including smart door lock control, guitar tuning, cycling interfaces, and simple lip-based games.
  • User experiments demonstrated that tactile feedback significantly improved interaction accuracy (from 41.3% to 66.3%) and optimized electrode point reduction to achieve 93%.

• What advantages does it have compared to existing solutions?

  • Supports both input and output on the same lip surface.
  • Offers good comfort without significantly interfering with users' daily activities (e.g., speaking or eating).
  • Flexible adherence to lips with relatively low production costs.

• What were the experimental or evaluation results?

  • Preliminary experiments showed users preferred interacting with the upper lip, which offered higher flexibility.
  • Co-located design with tactile feedback significantly improved accuracy, demonstrating the feasibility of tactile feedback in scenarios where eyes and hands are occupied.
  • Reduced electrode count design further enhanced device usability.

• Limitations and Future Directions

  • Limitations:
    • Electrotactile stimulation requires calibration (approximately 5 minutes).
    • Capacitive sensors may need recalibration due to saliva interference.
    • Certain lip-related activities (e.g., applying lipstick) may be restricted.
    • The device's appearance may be noticeable, potentially lacking social acceptance for some users.
  • Future Directions:
    • Explore further miniaturization and optimization of hardware implementation.
    • Design broader application scenarios tailored to diverse user needs and special requirements.
    • Conduct more comprehensive user experience studies to evaluate social acceptance and long-term usage performance.