On the Benefits of Image-Schematic Metaphors when Designing Mixed Reality Systems

Title of the Paper

The Benefits of Using Schema Metaphors in Mixed Reality System Design

Paper Information

• Subject Area: Human-Computer Interaction, Mixed Reality (MR) System Design, User Learning Facilitation
• Keywords: Schema Metaphors (ISM), Mixed Reality (MR), User Interface Design, Task Learning, Interaction Feedback, Novelty

Research Background and Issues

• Problems or Challenges:

  • Everyday users face high interaction demands when using mixed reality systems, including motor skills, spatial cognition, attention management, and understanding abstract computational concepts.
  • To achieve broader commercial applications, mixed reality systems need to be easy to learn and use while accommodating users with varying skill levels. However, large-scale adaptation incurs high costs.
  • Current mixed reality design guidelines are either too broad to provide specific implementation guidance or too low-level, focusing on micro-components while lacking intermediate methods to guide the construction of users' mental models.

• Research Motivation:

  • Schema metaphors (ISM) are cognitive structures derived from universal sensory experiences, linked to basic human sensory-motor experiences. Theoretically, they can reduce learning burdens through interaction designs that align with users' mental models.
  • Although previous research on ISM in screen-based GUIs has demonstrated its benefits for enhancing user experience, its practical impact in MR remains systematically unverified.

Solution

• Proposed Method:

  • The authors designed and implemented an enhanced mixed reality system based on schema metaphors (ISM), integrating 12 relevant metaphors into MR interface components and user interactions.
  • Experimental designs were used to compare the performance of ISM-enhanced systems with existing baseline systems.

• Innovations:

  • For the first time, schema metaphors were extensively integrated into mixed reality user interface design, covering multiple dimensions such as visual presentation, spatial distribution, and audio feedback.
  • A "system component—ISM—input/output instantiation" design pathway was proposed, providing a systematic method for subsequent ISM design and validation.

• Implementation Details:

  • The authors adopted 12 ISMs consistent with user behaviors in technical learning contexts and applied them to enhance existing MR system designs, such as video playback, tool menus, and interactive buttons.
  • Implementation steps included adjustments to the visual effects of the user interface (e.g., path-based redesign of video playback sliders) and interaction behaviors (e.g., press-based button designs).
  • User studies compared the enhanced system and baseline system across multiple dimensions, including task completion time, error rate, learning curve, and subjective experience.

Research Findings

• Key Results:

  • The ISM-enhanced system significantly improved users' task completion time, learning rate, and cognitive efficiency.
  • ISM notably increased subjective ratings of system comprehensibility, efficiency, and novelty.
  • Task completion time was reduced by over 25%, with error rates comparable to or slightly lower than the baseline system.

• Advantages over Existing Solutions:

  • Surpassed interface designs that merely adhere to guidelines, enhancing traditional efficiency metrics (e.g., speed and accuracy) while also improving non-goal-oriented qualities (e.g., novelty).
  • The strong physical immersion characteristics of MR amplified the cognitive facilitation effects of ISM, resulting in more significant learning improvements compared to screen-based interactions.

• Experimental or Evaluation Results:

  • Task completion time: The ISM group was significantly faster than the baseline group, with notable improvements for low-skill users.
  • Learning benefits: Learning rate (0.89 vs. 0.58) and learning gains were significantly higher than the baseline system.
  • Subjective experience (UEQ): Significantly better ratings in efficiency, comprehensibility, and novelty dimensions compared to the baseline.
  • Cognitive efficiency: Users enhanced by ISM demonstrated more efficient task performance under the same proportion of mental workload.

• Limitations and Future Directions:

  • The study validated specific task applications with 28 users; broader applicability requires verification across multiple scenarios and tasks.
  • The specific contribution and relative effectiveness of each ISM remain unclear.
  • Future research should explore the generalizability and independent contributions of ISM across different design tasks and interface types.

Conclusion

The study demonstrates that schema metaphors provide an effective intermediate method for designing MR systems that accommodate diverse user mental models while balancing broad applicability and cost efficiency. Future research should aim for broader validation and deeper exploration to determine whether ISM's generalizability consistently holds across contexts.