Grab It, While You Can: A VR Gesture Evaluation of a Co-Designed Traditional Narrative by Indigenous People

Title of the Paper

Grab It, While You Can: A VR Gesture Evaluation of a Co-Designed Traditional Narrative by Indigenous People

Paper Information

• Subject Area: Human-Computer Interaction (HCI), Virtual Reality (VR), Cultural Heritage Preservation
• Keywords: Gesture Interaction, Virtual Reality, Natural Interaction, Namibia, User Experience, Indigenous People, Diversity, Community Participation

Research Background and Issues

• Identified Problems and Challenges:

  • Current gesture interaction designs lack sufficient validation in practical applications; most studies fail to incorporate gestures into functional prototypes.
  • Participant diversity is insufficient, with a primary focus on participants from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) backgrounds.
  • Many gesture-related studies lack detailed descriptions of usage scenarios and experimental specifics, making experiments difficult to replicate, reproduce, or generalize.

• Significance of the Research:

  • Gesture interaction offers new means to embody and transmit intangible cultural heritage, especially in shaping immersive cultural experiences.
  • Including a broader range of participants helps reduce system bias and promotes the development of universally applicable designs.
  • Digitizing traditional culture aids in preserving endangered cultural practices and fostering their transmission.

• Motivation and Related Work:

  • Previous studies have shown that user-created gestures lack diversity and practical implementation validation.
  • Research on gesture-based cultural heritage representation in VR remains limited, with insufficient exploration of diverse natural interaction methods.

Solution

• Methods and Measures:

  • A community-based co-design approach was adopted in Namibia's Ju/’Hoansi community to digitize a traditional hunting story narrated by indigenous elders and implement it in VR.
  • The 7C model was used to progressively transform the story into a VR narrative.
  • Three mid-air gestures and one grabbing gesture were implemented, including "animal gesture," "crouching gesture," "wind direction testing gesture," and grabbing gesture, extracted from the community's hunting demonstrations.
  • These gestures were integrated into VR and validated through evaluations by community members.

• Innovations:

  • Combined real community co-design with indigenous bodily gesture practices to enhance the authenticity of digital cultural experiences.
  • Developed a comprehensive research framework from in-situ gesture elicitation to VR interaction validation, offering a new perspective for gesture interaction research.
  • Introduced non-WEIRD participant groups into the study to enhance cultural diversity and reduce cultural bias in design.

• Implementation Steps and Key Technologies:

  1. Gesture Elicitation: Observed and recorded natural gestures during on-site hunting demonstrations, identifying key interactive gestures through discussions.
  2. VR Narrative Design: Divided the story into modules according to its plot, implementing four scenes with gesture interactions.
  3. VR-Based User Evaluation: Returned to the community for on-site VR testing, analyzing users' gesture performances through recordings and observations.
  4. Data Processing and Analysis: Combined open coding and qualitative analysis to study users' gesture usage characteristics, interaction methods, and their variations.

Research Outcomes

• Specific Results:

  • Successfully implemented three mid-air gestures and one grabbing gesture in the VR application, revealing how users actually utilized these gestures.
  • Observed that users spontaneously employed some unelicited gestures, such as "spreading out," "approaching," and "simulating bow and arrow shooting."
  • Users generally gave positive feedback on the cultural authenticity and immersion of the VR application, with elders particularly noting that VR recreated the traditional hunting experience.

• Advantages Over Existing Solutions:

  • Proposed a more realistic gesture validation approach: from initial elicitation in natural settings to practical application in VR.
  • Expanded the contextual applicability of existing gesture classifications, providing a new case study for gesture research in the Global South.
  • Through co-design, indigenous people determined the digital representation of their culture, ensuring the authenticity of cultural reproduction.

• Experimental and Evaluation Results:

  • The success rate of mid-air gestures was high, with the "crouching" gesture widely accepted; however, the actual performance of "animal gesture" and "wind direction testing gesture" varied due to cultural differences among community members.
  • The study found that the lack of haptic feedback affected users' grabbing experiences; additionally, users expected the ability to grab multiple objects simultaneously.

• Limitations and Future Directions:

  • The low frequency of certain gestures (e.g., wind direction testing) used by community members suggests the need for additional contextual cues or triggers in VR.
  • The grabbing gesture should be further optimized to support multi-object grabbing and better simulate real-world interactions.
  • Future research should explore cross-cultural gesture study methods more broadly, including applications in other non-WEIRD communities.