Enabling Tangible Interaction on Non-touch Displays with Optical Mouse Sensor and Visible Light Communication

Title of the Paper

Enabling Tangible Interaction on Non-touch Displays with Optical Mouse Sensor and Visible Light Communication

Paper Information

• Field of Study: Human-Computer Interaction (HCI)
• Keywords: Tangible, Tabletop, Optical Mouse, Visible Light Communication, Tangible User Interfaces, Object Tracking, Display Interaction, VLC, Interaction Devices, Sensor Platforms

Research Background and Problem Statement

• Identified Problems or Challenges:

  1. Traditional Tangible User Interfaces (TUIs) require precise tracking of object properties (e.g., position and orientation), but existing solutions often involve expensive hardware and complex installation setups.
  2. Touchscreen-based tracking methods face the "grounding problem," where objects may not be trackable when not being touched or grounded by the user; additionally, their ability to track concurrent objects is limited.
  3. Technologies using additional devices such as projectors or depth cameras still face limitations in terms of applicability.

• Significance: Embedded tangible interaction interfaces integrated with display screens have broad application potential in areas such as education, gaming, and productivity tools. These interfaces enhance user interaction experiences, but low-cost, precise positioning, and hassle-free installation solutions are critical.

• Motivation and Related Work:

  1. This study leverages the optical sensing capabilities of optical mouse sensors and Visible Light Communication (VLC) technology to propose a novel low-cost tangible tracking method.
  2. Existing research on screen-based tracking technologies, such as infrared cameras, RFID sensors, and digital light processing projectors, has demonstrated successful applications but often lacks accessibility and affordability.

Proposed Solution

• Proposed Method:

  1. A system named "Centaur" was designed, combining Visible Light Communication technology and optical mouse sensors to enable tangible interaction on non-touch displays.
  2. The system dynamically transmits high-frequency light signals as position beacons via screen pixels, which are decoded by optical mouse sensors to determine the position and orientation of tangible objects.

• Innovations:

  1. Utilizes the pixel refresh rate characteristics of ordinary screens to transmit invisible light signals for precise tracking via mouse sensors, minimizing interference from traditional visual patterns.
  2. Proposes a cost-effective tracking method that supports tangible interaction using standard personal computers without requiring specialized hardware.
  3. Redefines the functionality of optical mouse sensors, employing them as receivers for VLC signal decoding rather than solely as displacement measurement devices.

• Implementation Steps and Key Technologies:

  1. On the Screen Side: Embeds a light signal transmission system, dividing the screen into multiple small regions and transmitting different position IDs.
  2. On the Mouse Side: Optical mouse sensors capture screen light signals without requiring additional sensors or hardware.
  3. Tracking System: Based on the collected light signals and position data, calculates the real-time position and orientation of objects; integrates mouse displacement tracking to address real-time tracking latency.

Research Outcomes

• Specific Results:

  1. Accurate Tracking Capability: Centaur can track the position and orientation of multiple optical mice or customized MouseTokens in real time on the screen.
  2. Flexibility: The system is compatible with both touch and non-touch screens, eliminating interaction barriers caused by the "grounding problem."
  3. Low Cost: Customized MouseTokens cost approximately $10 to manufacture, with the potential to further reduce costs using off-the-shelf commercial optical mice.

• Advantages over Existing Solutions:

  1. Compared to touch-based tangible interaction, Centaur addresses the issue of tracking failure when objects are not touched and supports more concurrent tangibles.
  2. Compared to camera/projector-based tracking methods, Centaur is simpler to operate, more cost-effective, and less dependent on specialized hardware.
  3. The combination of high-refresh-rate screens and optical mice minimizes visual interference.

• Experimental or Evaluation Results:

  1. Positioning Accuracy: Achieved a resolution of 12×12 mm, with excellent accuracy in application scenarios.
  2. Orientation Estimation Error: Average error within ±2 degrees, outperforming many orientation estimation methods based on peripheral patterns or touchscreens.
  3. Positioning Latency: Initial VLC positioning takes approximately 0.41 seconds, and orientation tracking takes about 0.8 seconds, comparable to other time-domain encoding methods.
  4. Interference Resistance: Under various screen content backgrounds (static images or dynamic videos), the statistical packet error rate is typically below 20%.

• Limitations and Future Directions:

  1. Limitations: For certain commercial optical mice, access to sensor registers may be restricted; positioning failures may require manual adjustment of the mouse position for retrying.
  2. Improvements:
     • Enhance the data sampling rate of mouse sensors to reduce positioning latency.
     • Utilize higher-resolution screens to improve absolute positioning accuracy.
     • Add forward error correction to further enhance positioning performance under dynamic content.
  3. Future Applications: Extend the technology to OLED screens, integrate with touchscreens and robotic systems, and provide more comprehensive interaction experiences and collaborative environments.

Markdown Formatting Notes

The scientific information is clear and well-organized, with distinct sections. Key information is not omitted. Sections without provided information are omitted.