"Grab the Chat and Stick It to My Wall": Understanding How Social VR Streamers Bridge Immersive VR Experiences with Streaming Audiences Outside VR

Research Background and Problem

• Identified Issues or Challenges: Social Virtual Reality (Social VR) offers a novel immersive interaction platform, but extending its interactive experience to audiences outside VR (e.g., via 2D live streaming platforms) remains underexplored. Furthermore, compared to traditional 2D live streaming, Social VR live streaming involves more complex technical requirements and social dynamics, encompassing interactions across physical and virtual environments.
• Significance: As an emerging practice, Social VR live streaming not only innovates online social interaction methods but also prompts a rethinking of the possibilities of cross-reality interactions. Addressing this issue can help create more inclusive, immersive, and safe cross-reality social environments.
• Research Motivation and Related Work: While existing research has focused on the dynamic interactions within Social VR and audience management in traditional live streaming, there is a lack of in-depth exploration of how to coordinate interactions between in-VR participants and external audiences in Social VR. This innovative practice requires a more detailed understanding and supportive design.

Solution

• Core Approach/Solution: Through semi-structured interviews with 17 experienced Social VR streamers, the study explores how they use innovative strategies to bridge immersive VR experiences and 2D platform audiences.
• Innovations:
  1. Identified four unique streamer strategies: real-time full-body and facial tracking, virtual camera control, haptic interactions, and role-playing.
  2. Highlighted the physicalized and tangible characteristics in cross-reality interactions.
  3. Introduced deeper levels of authenticity and emotional resonance beyond traditional live streaming.
• Implementation Steps and Key Technologies:
  1. Using full-body/facial tracking technology, streamers can naturally interact with audiences through body language and emotions.
  2. Leveraging virtual cameras in VR, streamers can simulate the audience's first-person perspective to create a more immersive viewing experience.
  3. Developing audience-triggered audio, visual, and haptic interaction mechanisms allows audiences to have a tangible impact on the streamer's virtual experience.
  4. Utilizing role-playing features in Social VR to collaborate with audiences in creating immersive scenario-based interactions.

Research Findings

• Specific Findings:
  • Identified four major strategies used by Social VR streamers:
    1. Real-time tracking of full-body and facial movements to dynamically express emotions and reactions.
    2. Innovative use of virtual cameras to enable audiences to participate from a first-person perspective.
    3. Allowing audiences to directly influence the streamer's virtual experience through haptic interactions.
    4. Building more personalized connections with audiences through role-playing.
  • Proposed directions for optimizing interaction design between Social VR and 2D live streaming platforms.
• Advantages Over Existing Solutions:
  • Compared to the limited text and non-verbal communication in traditional 2D live streaming, Social VR live streaming offers more authentic and immersive interactions.
  • Unique cross-reality interaction strategies enhance the audience's sense of presence and participation in virtual reality.
  • Emphasizes natural and tangible interactions over task-oriented designs commonly seen in traditional VR research.
• Experimental or Evaluation Results:
  1. Full-body tracking deepened interactions between streamers and audiences, with audiences perceiving stronger reactions from streamers.
  2. Virtual cameras and haptic interactions significantly enhanced audience engagement.
  3. Role-playing helped audiences better understand the streamer's authentic personality but also posed more complex management challenges for streamers.
• Limitations and Future Directions:
  • Limitations:
    1. Tracking systems may lead to privacy breaches or technical errors (e.g., tracking loss).
    2. Managing virtual cameras is complex, potentially increasing the streamer's burden and causing motion sickness for audiences.
    3. Excessive haptic interactions could disrupt the smoothness of the live streaming experience.
    4. Balancing interactions with audiences and in-VR users poses management difficulties.
  • Future Directions:
    1. Develop more efficient virtual camera control and tracking recovery mechanisms.
    2. Provide finer-grained privacy protection features for tracking.
    3. Introduce balancing and filtering mechanisms to manage the frequency of audience-triggered haptic interactions.
    4. Design cross-platform interaction tools to facilitate seamless collaboration between VR users and non-VR audiences.

In summary, this study not only provides theoretical support for Social VR live streaming practices but also offers clear implementation directions for designing next-generation cross-reality interaction platforms, supporting more complex yet safe and inclusive social dynamics.