Can AI Prompt Humans? Multimodal Agents Prompt Players’ Game Actions and Show Consequences to Raise Sustainability Awareness

Research Background and Issues

Problems or Challenges Identified by the Authors

• Long-term and Imperceptible Consequences: The consequences of unsustainable behaviors are often long-term and difficult to perceive, making it challenging for the public to intuitively understand their impact in daily life.
• Limitations of Traditional Games: Traditional serious games rely on predefined options and limited interaction methods, failing to fully reflect players' real-life intentions and actions. Additionally, these approaches may not effectively bridge the gap between virtual environments and real-life behaviors.

Why This Issue Is Important

• Necessity to Address Environmental Crises: As global climate change and environmental degradation intensify, fostering public awareness of sustainable behaviors becomes increasingly critical.
• Potential for Education and Influence: Games, as educational tools, can vividly demonstrate the consequences of real-world behaviors through role-playing and immediate feedback in virtual worlds, potentially inspiring behavioral change.

Research Motivation and Related Work

• New Opportunities with Large Language Models (LLMs): Advances in generative AI and large language models offer possibilities for simulating human behavioral consequences and enriching interactive experiences.
• Gaps in Existing Work: Although games aimed at raising sustainability awareness exist, they face challenges such as complex interaction mechanisms and high learning costs, limiting their appeal to a broader audience.

Solution

Proposed Method or Solution

• EcoEcho Game Design: Development of a generative AI-based virtual role-playing game that integrates multimodal agents (e.g., visual, auditory, and conversational) to enhance players' awareness of sustainability through real-time action-consequence mechanisms.

Innovations of the Solution

1. Generative AI-Based Dialogue System: Custom NPC interactions allow players to communicate with the system using natural language, with their behaviors directly mapped to in-game actions.
2. Action-Consequence Feedback: The game dynamically provides feedback on players' behaviors through gradually changing environmental visuals (e.g., increasing pollution levels).
3. Embedded Evaluation Features: Multi-stage voting mechanisms within the game capture real-time changes in players' attitudes toward sustainable energy policies.

Implementation Steps and Key Technologies

1. Game Narrative Design: Using the RECIPE framework, the narrative focuses on the UN Sustainable Development Goal of "Affordable and Clean Energy," enabling players to act across two eras through a time-travel storyline.
2. Multimodal Agents and Intent Detection:
   • NPCs are equipped with unique roles and behavioral logic using the Llama large language model and multimodal generation tools (e.g., DALL·E and Suno).
   • An intent detection system interprets players' language inputs into specific actions.
3. User Evaluation and Feedback Iteration:
   • Questionnaires, semi-structured interviews, and embedded game behavior tracking are employed to quantify the game's impact on awareness and behavior.

Research Outcomes

Specific Achievements

1. Significant Increase in Behavioral Intentions: Experimental results show that participants' scores on sustainable behavioral intentions (General Ecological Behavior, GEB) significantly improved after playing the game, though changes in ecological attitudes (New Ecological Paradigm, NEP) were minimal.
2. Game Promoted Multidimensional Thinking: Players reported in interviews that interacting with different NPCs enabled them to consider sustainability issues from various perspectives, such as social equity, technological innovation, and policy-making.
3. Potential Real-World Behavioral Impact: Many players mentioned that the game heightened their awareness of sustainable behaviors in real life, such as reducing plastic use and purchasing electric vehicles.

Advantages Compared to Existing Solutions

• Lower Learning Curve: Natural language interactions replaced complex mechanisms, attracting non-gamers to participate.
• Immediate Behavioral Feedback: Dynamic environmental changes reinforced the visibility of behavioral consequences, enhancing participants' depth of reflection.
• Personalized Narrative Experience: AI-driven NPCs enriched the immersion and diversity of the narrative.

Experimental or Evaluation Results

• Behavioral Changes: GEB scores significantly increased from 3.40 to 4.09, demonstrating the game's ability to influence behavioral intentions in the short term.
• Stable Attitudes: NEP scores showed a slight increase but did not reach statistical significance, suggesting that attitude changes may require longer-term interventions.

Limitations and Future Directions

1. Sample Representativeness: Participants were predominantly young, highly educated, and tech-savvy, potentially biasing the results.
2. Time Constraints: Short-term experiments cannot fully verify the game's impact on long-term behavioral changes.
3. Depth of Dialogue and Narrative: Players expressed a desire for deeper NPC interactions, with stronger connections to character backgrounds and core issues.
4. Need for Controlled Experiments: Future research should include more control groups to eliminate external factors influencing results.
5. Limitations of Generative AI: Challenges such as inconsistent quality in AI-generated visuals and narratives require further optimization.

Through the case study of EcoEcho, this research highlights the potential of generative AI in serious game-based education, while addressing challenges in human-computer interaction design, AI ethics, and sustainability. It provides valuable insights for future practices.