KeyTch: Combining the Keyboard with a Touchscreen for Rapid Command Selection on Toolbars

Document Title

KeyTch: Combining the Keyboard with a Touchscreen for Rapid Command Selection on Toolbars

Document Information

• Domain: Human-Computer Interaction (HCI), Interaction Techniques
• Keywords: Toolbar, Menu, Command Selection, Interaction Techniques, Keyboard-Touchscreen Integration, Object-to-Command Transition, Mobile Devices

Research Background and Problem

• Research Problems and Challenges:

  • Traditional desktop application toolbars occupy screen space and require users to frequently move the mouse pointer between the toolbar and the working object, causing interruptions in interaction flow and time inefficiency (referred to as "object-to-command transition").
  • While keyboard shortcuts are efficient, they demand users to memorize numerous commands, limiting their use to only the most common operations.
  • Existing methods to enhance input devices often require upgrading the mouse or keyboard, increasing user costs.

• Significance of the Problem:

  • Toolbars are critical interaction components in desktop applications; optimizing their design can improve operational efficiency and reduce time costs.
  • Enabling users to seamlessly access a large number of commands can significantly enhance the user experience in complex applications.

• Research Motivation:

  • To explore a more efficient method for toolbar command selection by integrating the keyboard and touchscreen, i.e., the KeyTch system.
  • To reduce the cost of object-to-command transitions and save screen space.
  • To leverage existing personal devices (e.g., smartphones) to enhance interaction capabilities.

Solution

• Proposed Solution:

  • The KeyTch system integrates the keyboard and touchscreen to display and select toolbar commands.
  • Command selection is achieved through a combination of key presses and touchscreen gestures.

• Innovations:

  1. Reduced Object-to-Command Transition Costs: The mouse pointer remains on the working object, minimizing frequent movements.
  2. Screen Space Saving: Toolbars are relocated to the touchscreen (personal device).
  3. Utilization of Existing Devices: Ordinary smartphones are used, eliminating the need for additional hardware.

• Implementation Steps and Key Techniques:

  1. Touchscreen Accessibility Study: Identifying the comfortable operating zone for users when combining the screen and keyboard.
  2. Interaction Technique Design:
     • KeyTch Touch (Direct Touch): Users directly tap toolbar commands on the touchscreen.
     • KeyTch Magnet (Indirect Gesture): Commands are selected via a virtual pointer with magnification.
     • KeyTch Key (Intermediate Key): Keyboard keys select the first-level toolbar, while touchscreen operations select the second level.
  3. Experimentation and Evaluation:
     • Comparing the above techniques with traditional mouse operations to evaluate completion time, accuracy, and user satisfaction.

Research Outcomes

• Specific Results:

  • Experiments confirmed that most 5.8-inch touchscreens can be comfortably operated by users, with performance surpassing that of the mouse.
  • KeyTch significantly reduced command selection time (40% reduction for single-level toolbars, 30% for two-level toolbars).
  • The system can manage a large number of commands (up to 720 commands) while maintaining high accuracy (over 95%).

• Advantages over Existing Solutions:

  • Utilizes ordinary devices (smartphones), avoiding costly upgrades.
  • Improves interaction fluidity by reducing switching time in mouse operations.
  • Supports more complex toolbar layouts and more efficient command selection.

• Experimental or Evaluation Results:

  • KeyTch Touch (KTT) achieved significantly shorter completion times than the mouse, with comparable accuracy (approximately 98%).
  • KeyTch's two-level toolbar technique outperformed the mouse in efficiency (approximately 30% reduction in completion time).
  • Subjective user evaluations indicated an overall positive experience with KeyTch, particularly excelling in scenarios without "quasi-modes."

• Limitations and Future Directions:

  • Limitations:
    • Learning Curve: Adapting to the new toolbar layout requires time.
    • Some users prefer familiar mouse operations.
  • Future Directions:
    • Exploring the possibility of dual-hand parallel use of KeyTch.
    • Extending KeyTch to handle more complex toolbar components (e.g., sliders, lists).
    • Investigating user memory and discoverability of toolbar commands.
    • Designing more ergonomic configurations, such as LCD keyboards or touchscreens integrated into desktop environments.

Comprehensive Summary

The KeyTch system innovatively combines the keyboard and touchscreen to provide an efficient alternative for toolbar command selection. Experimental results demonstrate significant reductions in task completion time while maintaining high accuracy, enabling users to enjoy a smoother interaction experience without additional hardware. By supporting large-scale command management, KeyTch offers valuable insights for future complex multitasking scenarios. Its potential extends beyond toolbar design, paving the way for new research in integrated touchscreen and keyboard interaction.