IEDS: Exploring an Intelli-Embodied Design Space Combining Designer, AR, and GAI to Support Industrial Conceptual Design

Research Background and Issues

• What problems or challenges did the authors identify?
  The authors highlighted numerous challenges faced during the conceptual design phase of industrial products, including limitations in design representation (physical and virtual) and inadequacies in design materials and tools. These issues may constrain designers' creativity and efficiency. Although augmented reality (AR) technology can enhance intuitive interaction and generative artificial intelligence (GAI) can offer diverse design solutions, there remains a gap in effectively integrating the two to support conceptual design.

• Why is this issue important?
  Industrial conceptual design is a critical phase in product development, typically accounting for 60-80% of development costs. Improving the efficiency and creativity of this stage is not only significant for the overall development process but also directly impacts the quality of the final product and its market responsiveness.

• Research Motivation and Related Work:
  Inspired by theories of "embodied cognition" and "generative capability" in human-computer interaction (HCI) and cognitive science, the authors propose integrating AR and GAI to construct an extended design space (IEDS). This aims to address the limitations of traditional design, such as cognitive overload and restricted creativity, while enhancing collaborative design and rapid iteration capabilities.

Solution

• What methods or solutions did the authors propose?
  The authors proposed a multimodal design system called the "Intelligent Embodied Design Space (IEDS)," which combines designers' physical and virtual interactions, AR's augmented capabilities, and GAI's generative and optimization abilities to support industrial product conceptual design.

• What are the innovative aspects of this solution?

  1. Embedding high-fidelity virtual design forms generated by GAI into physical environments using AR, creating a hybrid design space that combines real and virtual elements, supporting real-time interaction, and enhancing immersion and expressive capabilities.
  2. Leveraging GAI to understand designers' intentions, including physical prototypes and specific textual requirements, to generate more consistent design solutions.
  3. Integrating multiple AR methods (head-mounted display [HMD], handheld display [HHD], and spatial augmented reality [SAR]) to meet diverse interaction needs across different scenarios.

• What are the implementation steps and key technologies used?

  1. Theoretical Construction and Validation: Defining the characteristics and interaction mechanisms of IEDS through literature review and expert interviews.
  2. Tool Development: Developing three tools combining AR and GAI (GAI+HMD, GAI+HHD, and GAI+SAR) to support IEDS applications in various scenarios.
  3. User Research and Testing: Conducting nine design workshops (with 27 participants) to complete conceptual design tasks, collecting feedback on designers' interaction experiences with IEDS and its impact on design processes, creativity, and collaboration.
  4. Key Technologies:
     • Utilizing generative models like Stable Diffusion XL to provide high-quality design images.
     • Employing Unity and MRTK to support real-time modeling and rendering on HMD devices.
     • Using GRADIO web interfaces for convenient HHD interaction.

Research Outcomes

• What specific outcomes were achieved?

  1. Proposed and implemented the IEDS system, developing three AR-based tools (GAI+HMD, GAI+HHD, GAI+SAR).
  2. Systematically analyzed the advantages and limitations of IEDS and proposed preliminary application principles, including how to combine different AR methods to achieve better interaction effects.
  3. Found that IEDS effectively improved efficiency and creativity during the conceptual design phase, particularly enhancing designers' imaginative space in the initial design stage.

• What advantages does it have compared to existing solutions?

  1. Seamlessly transitions physical design prototypes to high-fidelity virtual prototypes.
  2. Enhances real-time interaction capabilities through the integration of GAI and AR, significantly reducing the time cost of externalizing designs.
  3. Provides multimodal input and output design modes, enhancing the naturalness and diversity of human-computer collaboration.

• What were the experimental or evaluation results?

  1. Designers reported that IEDS significantly lowered the threshold for expressing design ideas, especially for team members with less technical expertise.
  2. Accelerated version iteration speed, supporting multimodal input, including text, images, physical prototypes, and 3D modeling.
  3. AR's immersive experience and GAI's diversity significantly enhanced creativity, though the high-fidelity details generated during the early design phase could lead to design fixation.

• Limitations and Future Directions:

  1. GAI's application in IEDS lacks the ability to generate structural and engineering knowledge.
  2. GAI+SAR is less effective for structured design tasks.
  3. Integration between IEDS tools has not yet achieved a fusion of multiple AR methods; future efforts could focus on developing more comprehensive collaborative interaction tools.
  4. With advancements in GAI's multimodal capabilities, future work could integrate 3D generation features and couple physical and virtual interaction logic.

Conclusion

This paper not only introduces the innovative concept of IEDS but also validates its potential value and challenges in industrial conceptual design through experiments. It provides a theoretical foundation and practical direction for further integrating AR and GAI into the design space, promising to lead transformative exploration of design tools in the HCI field.