Cross, Dwell, or Pinch: Designing and Evaluating Around-Device Selection Methods for Unmodified Smartwatches

Research Background and Problem Statement

What Problems or Challenges Did the Authors Identify?

1. The limitations of small touchscreen displays on smartwatches (e.g., restricted expressiveness in target selection and issues with finger occlusion of screen content).
2. The shortcomings of current "Around-Device Interaction" technologies, including reliance on additional hardware and their inapplicability to commercial smartwatches.
3. A lack of systematic research on "target selection methods for around-device input."

Why Is This Problem Important?

1. The increasing use cases for smartwatches are severely constrained by interaction inefficiencies, which significantly affect user experience.
2. Providing convenient and efficient input methods not only addresses interaction issues in current products but also offers critical design guidelines for future wearable smart devices.

Research Motivation and Related Work

1. Research Motivation: Addressing two major gaps:
   • Achieving around-device input techniques without modifying smartwatch hardware.
   • Systematically studying target selection methods for "around-device interaction."
2. Related Work:
   • Existing interaction methods include single-hand gestures (e.g., Apple's double-tap gesture on the Apple Watch), physical input using knobs and bezels, and around-device gesture sensing. However, these methods either rely on auxiliary tools (e.g., radar, cameras) or are difficult to commercialize directly.

Proposed Solution

What Methods or Solutions Did the Authors Propose?

1. Development of a one-dimensional fingertip tracking and target selection system—SonarSelect, based on a single-microphone sonar system integrated into an unmodified smartwatch.
2. Definition and experimental validation of three target selection methods tailored for "around-device interaction":
   • Double-Crossing: Selecting a target by moving the cursor across the target boundary.
   • Dwelling: Selecting a target by dwelling the cursor over it for a set duration (500ms).
   • Pinching: Selecting a target by performing a "pinching" gesture with the thumb and index finger of the hand wearing the smartwatch.

What Are the Innovative Aspects of This Solution?

1. Hardware-Friendly: Utilizes the built-in microphone and speaker of the smartwatch, eliminating the need for hardware modifications and enabling direct implementation on commercial devices.
2. Open-Source Accessibility: Provides open-source code to support future researchers and developers, reducing technical barriers to further development.
3. Comprehensive Experimental Design: Two user studies systematically compared the performance of different selection methods in binary tasks and multi-target scenarios.

What Are the Implementation Steps and Key Technologies Used?

1. Technical Implementation:
   • Utilized an improved single-microphone sonar algorithm (based on existing LLAP technology) for one-dimensional fingertip tracking.
   • Enhanced noise resistance by adjusting algorithm hyperparameters and incorporating noise filters (e.g., One-Euro filter).
2. Experimental Studies:
   • Study 1: Compared the performance of three target selection methods (Double-Crossing, Dwelling, Pinching) in a one-dimensional binary target selection task.
   • Study 2: Investigated the performance of Double-Crossing and Dwelling in multi-target sequential tasks and examined the impact of haptic feedback on user performance and comfort.
3. Evaluation Metrics:
   • Objective Metrics: Throughput (TP), Movement Time (MT), Error Rate (ER), Target Re-Entry Count (TRE).
   • Subjective Metrics: Comfort and task load (NASA-TLX).

Research Outcomes

What Specific Results Were Achieved?

1. Technical Performance:
   • SonarSelect demonstrated high accuracy within the smartwatch's interaction area (0-10cm), with an error range of 3.45-4.53mm.
   • Double-Crossing outperformed other methods in binary selection tasks (highest throughput: 2.18 bps).
   • Dwelling excelled in multi-target scenarios (lowest error rate: 1.36%).
2. User Perception:
   • Dwelling was rated by users as the most intuitive and comfortable method (comfort score: 4.33/5).
   • Pinching was deemed the least comfortable and practical due to its high error rate and the inconvenience of requiring two-handed operation.

What Are the Advantages Compared to Existing Solutions?

1. Does not require hardware modifications or custom devices, reducing usage barriers and costs.
2. Innovatively introduces sonar technology into the interaction domain of smartwatches, expanding the scope of research on off-screen interactions.
3. Experimental results and user satisfaction indicate that SonarSelect matches or even surpasses other physical input methods (e.g., knobs or touch-sensitive bezels).

What Were the Experimental or Evaluation Results?

1. In binary tasks:
   • Double-Crossing achieved significantly higher throughput than other methods, though Dwelling was rated as more comfortable.
2. In multi-target tasks:
   • Dwelling had the lowest error rate and was better suited for complex selection tasks.
3. Haptic Feedback:
   • Improved user comfort but did not significantly enhance task performance.

Limitations and Future Directions

1. Limitations:
   • Compared to multi-microphone systems, SonarSelect is limited to one-dimensional input.
   • Pinching suffered from high error rates due to the complexity of two-handed coordination and has not been optimized.
2. Future Directions:
   • Extend to two-dimensional or higher-dimensional sonar tracking to support more complex interaction scenarios.
   • Integrate SonarSelect into smartwatch operating systems to validate its commercial feasibility.
   • Conduct more in-depth comparative studies with other existing interaction technologies (e.g., bezels, knobs) to further explore its advantages and disadvantages.

This study reliably demonstrates the feasibility of achieving "around-device interaction" on unmodified smartwatches and provides new insights for designing more natural and intuitive smartwatch interactions.