AutomataStage: an AR-mediated Creativity Support Tool for Hands-on Multidisciplinary Learning

Document Title

AutomataStage: an AR-mediated Creativity Support Tool for Hands-on Multidisciplinary Learning

Document Information

• Subject Area: Augmented Reality, STEAM Education, Creativity Support Tools
• Keywords: Hands-on learning, Multidisciplinary learning, STEAM, Creativity support tool, Learning tool, Video see-through system, Interactive Automata

Research Background and Problem

• Problem or Challenge: Current hands-on multidisciplinary learning faces the challenge of hardware and software separation, making exploration and integration processes difficult to connect. Additionally, there is a lack of tools that support students' creative thinking and active exploration.
• Importance: Students need to integrate fields such as science, technology, and art for creative exploration to enhance creativity, learning interest, and self-efficacy.
• Research Motivation: The authors aim to develop a tool that enables students to more conveniently engage in creative ideation, exploration, and production in a multidisciplinary learning environment while providing opportunities for feedback and iteration.

Solution

• Method or Solution: The authors propose "AutomataStage," an augmented reality-based creativity support tool. It combines video see-through technology, modular hardware, and generative design methods to assist students in creating interactive motion devices (Interactive Automata).
• Innovations:
  1. Bridging digital design and physical production seamlessly through augmented reality technology to connect hardware and software.
  2. Providing a visual programming method based on state transition diagrams to support the integration of interactive processes.
  3. Offering hardware and operational see-through functionality, enabling students to observe sensor and motor operations and their internal structures in real-time.
• Implementation Steps:
  1. Ideation and character creation (designing motion characters using sketches and low-cost materials).
  2. Exploring sensors and defining conditions.
  3. Generating, modifying, and simulating linkage mechanisms using real-scale guiding tools to create physical devices.
  4. Defining motor outputs.
  5. Writing interaction logic and testing execution using state transition diagrams.

Research Outcomes

• Specific Outcomes:
  • User studies validated that AutomataStage helps students create diverse interactive motion devices within a short time, enhancing their understanding of hardware, sensors, mechanisms, and programming.
  • Post-intervention, students' learning test scores increased from 14.66% to 83.19%, demonstrating the system's effectiveness in supporting knowledge acquisition.
• Advantages:
  1. The seamless bridge between manual creation and digital design lowers learning barriers and stimulates student creativity.
  2. Hardware and operational see-through features provide real-time feedback, aiding students in optimizing designs and iterating.
• Experiment and Evaluation Results:
  • User studies revealed that students could explore and customize complex hardware and software components, such as sensors and motors, thereby understanding their principles and operations.
  • Students were able to integrate components and debug code using state transition diagrams.
• Limitations and Future Directions:
  • The current system relies on AR markers for hardware tracking, which may be inconvenient; future improvements could include camera-based image recognition.
  • Enhancing hardware durability and network stability to support long-term educational use.
  • Adding features suitable for collaborative learning among multiple students and expanding course content.
  • Incorporating functionalities for advanced programming or complex interaction logic to broaden students' creative expression.

Conclusion

AutomataStage provides effective support for multidisciplinary learning, combining augmented reality and modular hardware to significantly lower learning barriers and shorten the path from ideation to outcomes. This research offers inspiration for the future development of interactive and motion design tools and anticipates its broad application in the education field.