SAWSense: Using Surface Acoustic Waves for Surface-bound Event Recognition

Title of the Paper

SAWSense: Using Surface Acoustic Waves for Surface-bound Event Recognition

Paper Information

• Research Area: Surface acoustic wave sensing technology for user touch event and activity recognition
• Keywords: Surface acoustic waves, touch detection, activity recognition, gesture interface, acoustics, sensing

Research Background and Problem

• Problems or Challenges:

  1. Existing vibration or acoustic sensors, such as accelerometers or microphones, cannot simultaneously achieve high bandwidth, low noise sensitivity, and long-range data acquisition.
  2. Sensors like accelerometers and seismometers have low sampling bandwidth, while traditional microphones are sensitive to environmental noise, hindering their use in noisy environments.
  3. Although touch and activity recognition technologies exist, they often focus on specific scenarios and involve high deployment costs.

• Research Significance: Exploring a novel sensing method that can robustly operate on various material surfaces, is immune to environmental noise, suitable for wide-band event recognition, and features low cost and low power consumption will bring innovation to applications such as user interfaces and smart homes.

• Motivation and Related Work:

  1. Surface acoustic waves (SAWs) are two-dimensional waves confined to the air-material boundary of a surface, avoiding interference from airborne noise and exhibiting low attenuation characteristics.
  2. Repurposed voice pickup units (VPUs) offer high bandwidth and can isolate environmental sounds, complementing existing sensors.

Solution

• Method or Solution:

  1. Utilize VPUs to capture surface acoustic waves for event detection and machine learning classification.
  2. Design an optimized signal processing and machine learning classification pipeline, including customized Mel-frequency cepstral coefficients (cMFCC).
  3. Employ data augmentation and feature transformation techniques to enhance cross-user and cross-material recognition performance.

• Innovations:

  1. Propose a novel sensing method that is low-cost, highly sensitive, and capable of long-range surface acoustic wave capture.
  2. Conduct experiments on various surfaces and complex geometries, demonstrating robustness and adaptability.
  3. Achieve high-accuracy touch and event recognition in complex acoustic environments without relying on expensive or cumbersome deployment.

• Implementation Steps and Key Techniques:

  1. Hardware: Use VPU sensors and custom PCBs to capture SAW signals.
  2. Signal Processing: Extract frequency-domain features using cMFCC and optimize frequency band distribution with energy analysis.
  3. Machine Learning:
     • Evaluate basic linear support vector machines (SVM), random forest classifiers, and simple multilayer perceptron (MLP) models;
     • Propose and validate data augmentation and cross-material feature mapping techniques.

Research Outcomes

• Specific Results:

  1. Developed a robust surface acoustic wave sensing system capable of recognizing various touch gestures, achieving over 97% classification accuracy for six trackpad gesture types.
  2. In a home environment, achieved over 99% accuracy in recognizing 16 kitchen activities (e.g., stirring, chopping, placing utensils).
  3. Successfully validated the system on irregular geometries or flexible materials (e.g., toy dragon and clothing cuffs).

• Comparison with Existing Solutions:

  • Compared to accelerometers, seismometers, and traditional microphones, the SAWSense sensor performs better in noisy environments. It also features higher bandwidth and, unlike traditional microphones, is not affected by external audio interference.

• Experimental or Evaluation Results:

  1. Single-channel experiments demonstrated a reliable detection range of over 1 meter; adding a second channel enabled gesture direction recognition.
  2. Data augmentation significantly improved cross-user gesture recognition accuracy (up to 97.6%).
  3. Feature transformation techniques tailored to different material properties improved cross-material performance from 59.2% to 98.0%.

• Limitations and Future Directions:

  1. With only a single VPU, simultaneous multiple surface events cannot be distinguished.
  2. SAWs can only detect touch/events directly occurring on the surface; multi-surface linkage requires additional deployment.
  3. Future considerations include:
     • Using multiple sensors for event separation.
     • Optimizing model performance on larger and more diverse datasets.
     • Testing in more complex scenarios (e.g., wearable devices during activities).