Telextiles: End-to-end Remote Transmission of Fabric Tactile Sensation

Document Title

Telextiles: End-to-end Remote Transmission of Fabric Tactile Sensation

Document Information

• Subject Area: Human-Computer Interaction (HCI), Haptic Technology, Fabric Tactile Perception
• Keywords: Tactile Perception, Texture Recognition, Self-supervised Learning, Haptic Feedback, Passive Haptic Display, Machine Learning, Haptic Display, Virtual Reality

Research Background and Problem Statement

• Identified Problems or Challenges:
  1. In online shopping, users cannot touch fabrics, making it difficult to judge the comfort and texture of clothing.
  2. Existing tactile sensors need to be integrated into stable devices, and their recognition capabilities may be limited by instability in human usage.
  3. Current haptic-driven devices can only display a limited number of patterns, making it challenging to handle tactile transmission for unknown fabrics.
• Significance: Transmitting fabric tactile perception to remote environments can enhance the online shopping experience and promote remote collaboration in the fashion and textile industries.
• Research Motivation and Related Work:
  • Several fabric texture recognition methods have been proposed, such as friction sensors, biomimetic tactile sensors, and optical tactile sensors. However, these primarily address classification problems for known fabrics and struggle to process data from unknown fabrics.
  • Existing haptic feedback devices, such as rotary or ultrasonic actuators, can display limited physical textures but lack the logic to dynamically select unknown patterns.

Solution

• Method or Solution: Telextiles is a system for remotely transmitting fabric tactile perception. It constructs a latent space reflecting the proximity of fabric tactile perception using self-supervised contrastive learning, combined with tactile sensing devices and rotary-type actuators.
• Innovations:
  1. Training an encoder with self-supervised contrastive loss to learn the latent space of fabric tactile perception.
  2. Eliminating the need for user-dependent data collection devices (custom fixtures with stable pressure and angles).
  3. A rotary actuator capable of dynamically displaying the closest physical fabric sample.
• Implementation Steps and Key Technologies:
  1. Training Phase: Manufacturers collect tactile data from a large number of fabric samples and train the encoder to create the latent space.
  2. Transmission Phase: User A collects fabric data using a tactile sensor and uploads it to the server. The server extracts latent features and identifies the closest known sample, transmitting its ID or features to User B.
  3. The system uses self-supervised contrastive learning (MoCo) for training to ensure fabric similarity is reflected in the latent space.
  4. The haptic actuator rotates to present the most matching physical fabric sample.

Research Outcomes

• Specific Outcomes:
  1. Developed a latent representation based on self-supervised learning that reflects the proximity of fabric tactile characteristics.
  2. Designed a DIGIT optical tactile sensing device with a custom fixture for more stable and accurate tactile data collection.
  3. Created a rotary actuator capable of selecting and displaying the physical fabric sample closest to the input tactile characteristics.
• Advantages over Existing Solutions:
  • Significantly improved handling of tactile data from unknown fabrics.
  • The system can dynamically transmit and display unlimited fabric tactile characteristics, rather than being limited to known patterns.
  • The data collection process achieves higher stability and consistency.
• Experimental or Evaluation Results:
  1. Using k-nearest neighbor clustering for evaluation, the latent space achieved an 80.44% accuracy in fabric classification.
  2. The custom fixture improved tactile data collection stability, raising accuracy from 82.84% to 99.69%.
  3. User testing revealed some discrepancies between model selections and human judgments, potentially due to insufficient capture of sliding and friction tactile information.
• Limitations and Future Directions:
  • Limitations:
    1. Model selections in user testing did not fully align with human judgments.
    2. The DIGIT sensor did not completely capture tactile characteristics such as friction and thermal sensations.
    3. The rotary actuator can only display tactile information for a limited number of samples.
  • Future Directions:
    1. Further research on sensing methods to capture fabric friction and dynamic sliding sensations.
    2. Enhancing haptic actuators, such as supporting the display of 2D/3D material spaces.
    3. Integrating robotic platforms for automated fabric tactile data collection.
    4. Reducing actuator size to enable more portable remote tactile interactions, such as wearable devices like watches or fingertip devices.

References

• Includes extensive related work and technical background, such as tools for fabric cognition, tactile sensing devices, and classic algorithms related to self-supervised learning (e.g., MoCo, SimCLR).