MineXR: Mining Personalized Extended Reality Interfaces

Document Title

MineXR: Mining Personalized Extended Reality Interfaces

Document Information

• Subject Area: User interface design and personalization research in Extended Reality (XR)
• Keywords: Extended Reality, personalized user interface, dataset, interface layout, user behavior, interaction design

Research Background and Issues

• Identified Problems or Challenges:

  1. Difficulty in obtaining user behavior data and interface preferences from daily XR device usage.
  2. Current research on XR interaction and layout is primarily based on predefined content, lacking real user-generated personalized interfaces.
  3. It remains debatable whether application paradigms from traditional desktop and mobile devices are applicable to XR interfaces.

• Significance of the Research:

  1. XR has the potential to revolutionize user interaction with digital content, but a deep understanding of user preferences, activity contexts, and interface interactions is required.
  2. Supporting personalized and more efficient XR layouts could be a core driver for the adoption of next-generation XR devices.

• Motivation and Related Work:

  1. The study introduces design mining techniques, such as reusing web and mobile application interfaces, but these techniques have not been widely applied in the XR domain.
  2. Existing research includes interaction design tools and immersive prototyping tools, but these focus more on content presentation rather than the content and functionality users need.

Solution

• Proposed Approach:

  1. Developed the MineXR platform to collect and analyze personalized XR interfaces.
  2. Created user content-based personalized XR widgets using smartphones and head-mounted devices (HMDs).
  3. Provided an open large-scale dataset comprising user-generated XR layout data.

• Innovations:

  1. New Data Collection Method: Users capture application screenshots in their real-life environments and generate personalized XR components.
  2. Open Data Construction: The dataset includes 109 XR layouts and 695 widgets generated by 31 participants, addressing gaps in personalized XR interface research.
  3. Data mining and analysis tools enable researchers to derive insights from behavioral data, supporting context-aware XR systems.

• Implementation Steps and Techniques:

  1. Widget Generation: Users take application screenshots via a smartphone, crop functional blocks from the apps, and convert them into XR widgets.
  2. Widget Placement: Users specify widget positions in XR layouts using mobile devices and preview the results in real time.
  3. Data Storage and Analysis: Cloud technologies (Firebase and Azure Spatial Anchors) are used to store user layout and functionality data in real time.
  4. Data Annotation and Clustering: User data can be labeled via a web interface, including widget functional categories, contextual information, and UI element types.

Research Outcomes

• Specific Results:

  1. Generated an open dataset containing 109 XR layouts created by 31 users in four types of environments, covering a wide range of scenarios and tasks.
  2. Offered XR interface design recommendations, including functional widget construction and layout adaptation based on environments and activities.

• Comparison with Existing Solutions and Advantages:

  1. Unlike previous studies that only showcase predefined interfaces, MineXR allows users to create deeply personalized XR layouts, enhancing the study's realism and contextual diversity.
  2. The dataset distribution demonstrates higher external validity, making it suitable for developing more adaptive future XR systems.

• Experimental and Evaluation Results:

  1. Data shows that users prefer generating interfaces by functional decomposition rather than using entire applications, with 52.81% of widgets derived from screenshot cropping.
  2. Data reveals the distribution of widget categories across different contexts and their dependency on the environment. For example, 35% of widgets in kitchen environments come from food and beverage applications, while 54% in office environments come from productivity tools.
  3. Annotation analysis indicates that users frequently cluster widgets based on semantics and functionality, dynamically adjusting layouts according to activity needs.

• Limitations and Future Directions:

  1. The current dataset supports only static layouts; future research could explore temporal dynamic layouts and user-centered layouts.
  2. Fully interactive virtual elements are not yet integrated, and further studies need to incorporate long-term user behavior testing.
  3. Usage scenarios are primarily limited to laboratory and controlled environments; future work could extend to real-world studies with longer time spans and more complex contexts.

Conclusion

MineXR provides critical support for XR system design by collecting personalized XR interaction data. Its data-driven research approach will drive the development of functional XR interfaces and adaptive system designs. This study offers a robust tool and data foundation for future XR developers and researchers.