Exploring Multimodal Generative AI for Education through Co-design Workshops with Students

Research Background and Issues

• What problems or challenges did the authors identify?
  How students design and prototype future educational tools integrating multimodal large language models (MLLMs) has not been sufficiently explored. Existing educational technology tools often fail to adequately consider students' needs and learning experiences. Additionally, the impact and potential of multimodal generative AI in education remain unclear, including how to address learning difficulties, enhance feedback in educational tools, and improve personalization.

• Why is this issue important?
  The rapid development of generative AI, particularly MLLMs that integrate multimodal inputs, brings significant possibilities to the education field, including personalized learning resources, multimodal learning experiences, and real-time feedback. However, this also creates an urgent need to design these tools to meet students' needs. Ensuring that generative AI technologies are student-centered can enhance their effectiveness and adaptability in educational applications.

• Research Motivation and Related Work
  The primary motivation of this study is to incorporate students' perspectives into the design of educational tools and to explore the potential of MLLMs in education through co-creation workshops. Related research has explored the co-creation process between teachers and students in the intersection of generative AI and education but has paid less attention to student-designed educational AI tools.

Solution

• What methods or solutions did the authors propose?
  The authors conducted two co-creation workshops ("Creative Workshop" and "Application Design Workshop") to understand how students design and prototype future educational applications integrating MLLMs. These workshops included team collaboration, brainstorming, learning journey mapping, storyboard design, low-fidelity prototyping, and evaluation based on design principles.

• What are the innovative aspects of this solution?

  1. Student-Centered Approach: By directly involving students in the design process, the tools created are more likely to meet students' actual needs.
  2. Embedded Design Principles: The introduction of unique generative AI design principles (e.g., generative diversity, co-creation, embracing imperfection) to guide student creations.
  3. Focus on Multimodal Interaction: Encouraging students to consider how multimodal inputs (e.g., audio, video, text) can be integrated into educational applications.
  4. Design Tools and Output Evaluation: Using "learning journey maps" and "prompt flow diagrams" to structurally capture the characteristics of student-designed applications.

• What are the implementation steps and key techniques used?

  1. In the Creative Workshop, students identified learning challenges and conceptualized solutions using MLLMs, completing "future learning journey maps."
  2. In the Application Design Workshop, students were introduced to generative AI design principles and tasked with designing storyboards, prompt flow diagrams, and user interface prototypes while evaluating the interactivity and functionality of their applications.
  3. Design principle worksheets were used to help students reflect from a generative AI design perspective, exploring ways to improve the credibility and adaptability of AI outputs.

Research Outcomes

• What specific outcomes were achieved?

  1. Students designed multimodal AI application prototypes aimed at addressing educational challenges such as learning acquisition, skill development, resource management, time management, and classroom interaction.
  2. The application designs primarily focused on multimodal content generation and personalization, enhanced feedback, and emotion and behavior monitoring.
  3. A range of user-MLLM interaction perspectives were provided, including input (control methods) and output (feedback types) designs, reflecting students' deep thinking about generative AI design.

• What are the advantages compared to existing solutions?

  1. Multimodal Integration: Compared to solutions that only support text or single modalities, multimodal content generation and personalization better meet modern learning needs.
  2. Problem-Oriented with Student Insights: The solutions are directly rooted in real problems identified by students, making them more practical and targeted.
  3. Interaction Optimization: New interaction mechanisms were designed, enabling users to actively refine AI outputs and provide feedback on AI-generated content.

• What are the experimental or evaluation results?
  The student-designed application concepts validated the following perspectives:

  • MLLMs can generate diverse content formats, such as videos, games, and interactive learning resources.
  • Students believe GenAI can effectively improve learning outcomes, time management, and teaching interactions.
  • Student teams unanimously agreed that GenAI applications require effective feedback and control mechanisms to enhance the accuracy of generated content.

• Limitations and Future Directions

  1. Limitations:
     • The study had a limited sample size, involving only students from certain South Asian regions, which restricts the generalizability of the findings.
     • The student-designed applications focused more on opportunities than potential risks, such as issues with content accuracy.
  2. Future Directions:
     • Conduct larger-scale workshops with participants from diverse backgrounds to enhance applicability.
     • Explore how to integrate metacognitive support with design frameworks to guide the effective integration of generative AI in learning environments.
     • Investigate ethical issues and design challenges of GenAI in education, such as bias or privacy concerns.

This research provides valuable insights into the design and application of generative AI in the education field while highlighting potential areas for improvement. This collaborative design model aligns more closely with actual user needs, offering guidance and inspiration for the development of educational tools.