Designing Immersive, Narrative-Based Interfaces to Guide Outdoor Learning

Title of the Paper

Designing Immersive, Narrative-Based Interfaces to Guide Outdoor Learning

Paper Information

• Subject Area: Human-Computer Interaction, Educational Technology, Design Combining Augmented Reality and Outdoor Learning
• Keywords: Augmented Reality, Outdoor Learning, Machine Learning, Immersive Technology, Mobile Computing, Ubiquitous Computing, Computer Vision, Narrative Learning

Research Background and Problem

• Identified Problem or Challenge:
  While outdoor learning experiences (such as field trips) can improve children's scientific achievements and interest in learning, these activities are often difficult to implement, especially under time and curriculum constraints. Additionally, although narrative learning and technology can enhance the structure and engagement of learning, research on effectively integrating outdoor learning, narrative, and immersive technologies remains limited.

• Importance:
  Outdoor natural environments are ideal places for teaching scientific knowledge (such as biology and ecology), as they can make abstract concepts more tangible. This learning approach not only enhances students' interest in science but also fosters emotional connections with nature.

• Research Motivation and Related Work:
  Previous studies have shown that narrative learning can improve engagement and learning outcomes; augmented reality (AR) and context-aware technologies can provide more interactive and immersive learning experiences. However, there is still a lack of research integrating narrative, AR, and outdoor learning technologies. This study aims to fill this research gap.

Solution

• Proposed Solution:
  The authors developed a narrative-based mobile application called "Kiki's Adventure," aimed at guiding fourth to sixth-grade students in learning how to interact with their local environment. In one variant of the application, augmented reality (AR) and image classification technologies were incorporated to explore the value of these features.

• Innovative Aspects:
  By combining narrative guidance and technological support, scientific learning content is embedded into exploration activities related to the local environment. The introduction of virtual characters (such as Kiki) enhances students' emotional engagement, while computer vision and augmented reality provide functionalities for interacting with the real world.

• Implementation Steps and Key Technologies:

  1. Development of two application variants: a narrative-only version and a version combining computer vision and augmented reality (Narrative+CV+AR).
  2. Design of the application using the Unity game engine, incorporating image classification and augmented reality technologies (e.g., Niantic Lightship plugin).
  3. Core features of the application include natural language processing (keyword detection), free-response questions, AR activities (such as tree height measurement and virtual planting), and a tree species recognition image classification model.
  4. Training a ResNet-18 model using reference materials to classify tree photos, providing personalized feedback based on user interactions.

Research Outcomes

• Specific Outcomes:

  1. The application was proven effective in guiding students in outdoor learning and promoting learning outcomes during experiments.
  2. Participants expressed high overall satisfaction with the application and learning experience, particularly enjoying the narrative characters, hands-on field activities, and targeted question responses.

• Advantages Over Existing Solutions:

  • Achieved context-aware outdoor learning interaction without the need for complex equipment markers;
  • Enhanced student interest in learning through narrative elements;
  • Provided an efficient and user-friendly educational technology platform to connect nature with learning content.

• Experimental or Evaluation Results:
  User testing involving 44 children revealed:

  1. Significant improvement in scientific knowledge test scores after the course;
  2. The outdoor learning application increased emotional attitudes toward protecting nature and reduced attitudes supporting the exploitation of nature (e.g., endorsing tree cutting);
  3. Learning outcomes between the two application variants showed no significant differences, but the narrative-only version had a more pronounced impact on improving scientific attitudes.

• Limitations and Future Directions:

  1. Current AR and computer vision technologies are primarily entertainment-focused; future research should explore deeper connections with learning content.
  2. The study sample was limited to participants with high scientific interest; expanding the scope to include non-science-oriented groups is recommended.
  3. Tools suitable for other geographic environments and content need to be developed to achieve global adaptability and scalability.

Conclusion

By designing and testing two narrative-driven outdoor teaching applications, this study demonstrates how narrative, AR, and computer vision technologies can be effectively combined to guide scientific learning in local environments. It also proposes useful design principles for future development.