Where Should We Put It? Layout and Placement Strategies of Documents in Augmented Reality for Collaborative Sensemaking

Document Title

Where Should We Put It? Layout and Placement Strategies of Documents in Augmented Reality for Collaborative Sensemaking

Document Information

• Subject Area: Augmented Reality (AR), Collaborative Cognition, and Information Layout
• Keywords: Spatiality, Spatial Layout, Content Organization, Sensemaking, Visual Interaction, Qualitative User Study, Augmented Reality, Mixed Reality, Collaborative Sensemaking

Research Background and Issues

• Problems and Challenges:

  • In augmented reality (AR) office scenarios, the placement of virtual documents is difficult to standardize, especially regarding how placement and layout affect collaborative and personalized information processing.
  • It remains unclear how physical environments (e.g., furniture) and work styles (collaborative vs. individual) influence virtual content organization in sensemaking tasks.

• Significance:

  • As the potential applications of AR technology in future office environments are widely discussed, addressing these issues can better support brainstorming and information processing tasks, making AR more adaptable to real-world collaborative workflows.

• Motivation and Related Work:

  • The authors note that prior research has focused more on traditional screen layouts, desktop interactions, and virtual reality (VR) application scenarios, without delving deeply into layout strategies in collaborative AR contexts.
  • Related work suggests that layout design and the semantic characteristics of physical environments significantly impact collaborative activities and sensemaking tasks, but more detailed analysis and validation are needed.

Solution

• Method/Approach:

  • The authors conducted a mixed-design user study involving two types of physical office environments (fully furnished vs. sparsely furnished) and two work modes (collaborative vs. individual).
  • Using augmented reality headsets (HoloLens 2), they tested data classification tasks to analyze users' content layout and arrangement strategies.

• Innovations:

  • Proposed and analyzed layout and placement strategies for virtual content in AR scenarios involving multi-user collaboration and individual work.
  • Explored the relationship and impact of physical environments on layout decisions, with a particular focus on the semantic properties of furniture (e.g., the interactive characteristics of tables and whiteboards).

• Implementation Steps and Techniques:

  1. Experimental Design: Combined experiments with two environments (fully furnished room and sparsely furnished room) and two work modes (collaborative or individual tasks).
  2. Data Recording: Recorded users' content arrangement and interactions with physical furniture in AR spaces, followed by post-task interviews.
  3. Data Analysis: Used qualitative research tools to analyze users' layout strategies, interaction patterns, and environmental influences.

Research Outcomes

• Specific Findings:

  1. Workflow and Layout Strategies:
     • The workflow for sensemaking tasks consisted of planning, structuring, optimizing, and final presentation phases, with consistent layout decisions observed across workflows.
     • Furniture (especially tables and whiteboards) was frequently utilized under all conditions, while walls and other empty spaces were more commonly used in sparsely furnished environments.
  2. Collaboration vs. Individual Differences:
     • In collaborative tasks, users preferred furniture-based layouts, while individual tasks exhibited more diverse layouts.
     • Furniture positively contributed to defining collaboration boundaries and segmenting workflows.

• Comparison with Existing Solutions:

  • The study places a strong emphasis on the influence of physical environments on AR document layouts, offering in-depth exploration of collaborative and individual sensemaking tasks in AR scenarios.
  • It complements prior research focused on VR while addressing the gap in AR-specific studies.

• Experimental Evaluation Results:

  • In fully furnished rooms, teams significantly utilized furniture more and tended to create well-structured layouts; in sparsely furnished conditions, users more frequently used walls or floors.
  • Highly relevant document classification tasks encouraged semantic layout behaviors (e.g., spatially representing parent-child relationships).

• Limitations and Future Directions:

  • Limitations:
    • The limited field of view and tracking precision of HoloLens 2 affected user experience.
    • Task design restricted the exploration of creative scenarios (e.g., brainstorming tasks).
  • Future Directions:
    • Further exploration of collaborative dynamics, such as remote vs. co-located task modes.
    • Integration of semantic associations (e.g., document content and physical furniture properties) into future AR application designs.

Design Recommendations

1. AR Office Scenario Suggestions:

   • Provide ample furniture (e.g., tables and whiteboards) and blank surfaces for attaching and placing virtual content.
   • Consider selecting materials and surfaces that enhance user interaction intuitively through a sense of "realism."

2. Virtual Content Design Suggestions:

   • Support automatic layout features that generate natural arrangements based on furniture surface properties.
   • Provide visual annotation and linking tools to enhance document categorization and collaborative visibility.

3. Opportunities for Physical Environment Design:

   • Combine semantic layouts with virtual content, such as using physical scene locations to represent data relationships.
   • Leverage physical environments for asynchronous collaboration and result presentation, expanding indoor space utilization.