An Artists' Perspectives on Natural Interactions for Virtual Reality 3D Sketching

Title of the Paper

An Artists’ Perspectives on Natural Interactions for Virtual Reality 3D Sketching

Paper Information

• Subject Area: Human-Computer Interaction and Virtual Reality Art Creation
• Keywords: Virtual Reality, Gestures, Speech, Multimodal Interaction, 3D Sketching

Research Background and Issues

• Identified Challenges:

  • Current 3D Virtual Reality (VR) art creation tools are often limited to a single input mode (e.g., handheld controllers), failing to fully leverage the immersive nature of VR.
  • There is a lack of in-depth research into the needs and preferences of professional artists to improve VR 3D sketching systems.
  • Compared to traditional 2D drawing, 3D sketching is more complex, requiring artists to handle depth, space, and intricate perspective relationships, which poses higher demands on interaction design.

• Significance:

  • Enhancing the natural interactivity of 3D sketching applications can reduce cognitive load, increase tool intuitiveness, and enhance immersion.
  • Improving the compatibility of digital creation tools with artists’ needs can help them better express their creativity and expand VR’s applications in the creative design domain.

• Research Motivation and Related Work:

  • By evaluating the tools and interaction modes of existing commercial 3D sketching systems, the authors highlight the importance of developing natural unimodal and multimodal interaction techniques.
  • Related studies have primarily focused on single-mode interactions (e.g., gesture or voice input) but have rarely addressed artists’ preferences for multimodal interaction.

Proposed Solution

• Proposed Methods:

  • Design natural interaction methods from the perspective of artists (e.g., incorporating gestures, voice, bimanual interaction, etc.) and explore improvements to existing tools.
  • Collect feedback and improvement suggestions from artists on current 3D sketching systems through semi-structured interviews and experimental studies.

• Innovative Aspects:

  • Promote multimodal interaction by integrating gestures, voice, eye tracking, and other inputs into VR 3D sketching.
  • Propose specific tool optimization suggestions, such as "disappearing menus" and "simplified selection tools," to enhance user experience and interaction efficiency.
  • Transfer traditional 2D drawing functionalities and creative habits into the 3D VR environment.

• Implementation Steps and Techniques:

  • Use OpenBrush (an open-source VR painting tool) for experimental design.
  • Interview 13 art students to qualitatively analyze their interaction habits and needs during the creative process.
  • Conduct a comprehensive analysis of users’ physical actions, voice commands, and thought processes, combined with quantitative usability evaluation using the System Usability Scale (SUS).

Research Outcomes

• Specific Findings:

  • Multimodal interaction techniques (e.g., combining gestures with voice selection), realistic geometric shape generation, and continuity of artistic habits significantly improve the user experience of VR 3D sketching.
  • Identified specific needs of artists for tools, such as shape generation and filling, line beautification, and stroke segmentation for advanced functionalities.

• Advantages:

  • Compared to existing unimodal interaction systems, multimodal interaction more intuitively meets artists’ creative needs, enhancing user immersion and productivity.
  • The proposed improvements are tailored for artists rather than general users, providing professional support for VR art creation.

• Experimental and Evaluation Results:

  • Using the System Usability Scale (SUS), OpenBrush achieved an average score of 77 (out of 100), indicating above-average user experience.
  • Qualitative analysis revealed diverse suggestions from artists regarding tool functionalities and interaction design, such as incorporating voice, gestures, bimanual input, and even AI-assisted creation.

• Limitations and Future Directions:

  • The participant age range (20-28 years) and their VR experience levels limit the ability to fully reflect the needs of professional artists.
  • The short experimental duration suggests that future studies should extend drawing time and adopt multi-phase experiments to improve data quality.
  • Future research should include artists from diverse cultural and experiential backgrounds to explore the impact of regional or age differences on interaction preferences.

Summary and Recommendations

• Key Recommendations:

  • Further develop VR hardware to better accommodate the personalized needs of artists, such as controllers of different sizes or for specific purposes.
  • Incorporate natural interaction methods like voice, gestures, and eye tracking, allowing artists to choose based on personal preferences.
  • Develop functionalities similar to desktop drawing tools (e.g., shape generation, beautification, segmentation) to better support artists’ creative workflows.

• Future Directions:

  • Long-term accumulation of artist user data to gain deeper insights into the optimal application scenarios of multimodal interaction technologies in VR creative arts.
  • Explore AI-based creative assistance tools to enhance artists’ ability to express creativity in 3D sketching.