BackTrack: 2D Back-of-device Interaction Through Front Touchscreen

Title of the Paper

BackTrack: 2D Back-of-device Interaction Through Front Touchscreen

Paper Information

• Domain: Human-Computer Interaction (HCI) and smartphone touch interaction
• Keywords: Back-of-device interaction, capacitive sensing, smartphone touch, 2D trackpad, user interface design, human-machine interface, passive interaction device, prototype design, low-power interaction device, multi-touch

Research Background and Issues

• Identified Problems or Challenges:

  • The front touchscreen of smartphones is often obstructed by fingers, negatively impacting user interaction experience.
  • Back-of-Device (BoD) interaction utilizes idle fingers on the back of smartphones for operation, but existing systems are mostly limited to 1D sliding or tap interactions.
  • Most BoD implementations require external power sources or modifications to the operating system, making their application complex.
  • Lack of switching mechanisms leads to accidental touches and unintended operations.

• Significance:

  • Advancing BoD interaction technology can enhance smartphone operation flexibility, reduce screen obstructions, and improve user experience.

• Research Motivation and Related Work:

  • Existing BoD interaction methods (e.g., using cameras, microphones, or inertial measurement units) are susceptible to environmental noise and cannot achieve precise 2D finger trajectory tracking.
  • Some capacitive touch-based solutions require operating system modifications and lack mechanisms to prevent accidental touches.

Solution

• Method or Solution:

  • A system named "BackTrack" is proposed, transforming the back of smartphones into a passive 2D trackpad by sensing fine finger movements on the front capacitive touchscreen.
  • The system features a fully encapsulated design integrated into a phone case, requiring no additional power, wireless connection, or system modifications.

• Innovations:

  • Utilizes the multi-touch functionality of existing capacitive screens and designs a novel 2D trackpad encoding method that requires only 2N wire connections to achieve N×N resolution.
  • Employs transparent indium tin oxide (ITO) film to cover the screen, ensuring the functionality of the front touchscreen remains unaffected.
  • Introduces a unique battery-free switch mechanism activated or deactivated by thumb movements, preventing accidental touches.

• Implementation Steps and Key Technologies:

  1. Trackpad Electrode Design: Optimized electrode shape (circular), size, and spacing to balance capacitive changes and touch reliability.
  2. Electrode Connection Mode: Compared "grid," "rotated grid," and "zigzag" patterns for robustness, ultimately selecting the "grid pattern."
  3. Capacitive Switch Addition: Used a small copper foil as a switch, altering the capacitance between the user and the phone ground to enable switching functionality.
  4. Prototype Development: Integrated the above designs into a custom phone case compatible with various smartphone models.

Research Outcomes

• Specific Results:

  • Successfully implemented a 2D BoD trackpad system capable of real-time precise tracking of back-finger movements.
  • The system is entirely passive, requires no modifications to smartphone operating systems, and does not affect touchscreen functionality after installation.

• Advantages Compared to Existing Solutions:

  • Achieved a portable BoD interaction device independent of power sources and operating systems.
  • Enabled continuous 2D fingertip trajectory tracking, surpassing limitations of 1D or simple tap interactions.
  • Provided a hardware switch mechanism that significantly reduced accidental touch risks.

• Experimental or Evaluation Results:

  • Human trials showed that after only 3 minutes of training, users significantly improved accuracy across four interaction tasks:
    • Accuracy for 4-direction tapping reached 99.16%.
    • Accuracy for 4-direction sliding reached 100%.
  • Users reported no degradation in touchscreen performance due to the added ITO coating, even during high-precision tasks such as text input.

• Limitations and Future Directions:

  • Limitations:
    • System performance declines when users have moist fingers or wear insulating gloves (e.g., latex gloves).
    • Does not support multi-touch interactions and lacks tactile feedback.
    • Electrode design parameter optimization remains qualitative, lacking precise quantification.
  • Future Research Directions:
    • Develop quantitative models for BackTrack design and establish a systematic design framework.
    • Investigate the impact of user conditions (e.g., skin moisture and finger size) on signal-to-noise ratio and interaction performance.
    • Explore performance under varying environments and grounding conditions.

Additional Information

• The prototype product was adapted for multiple smartphone models, including Google Pixel 2, LG Nexus 5, Samsung Galaxy S8, and iPhone 11 Pro Max.
• The research was funded by Snap Inc., with partial support from the U.S. National Science Foundation (NSF).