Thought Bubbles: A Proxy into Players' Mental Model Development

Title of the Paper

Thought Bubbles: A Proxy into Players’ Mental Model Development

Paper Information

• Subject Area: Research on mental models in human-computer interaction, particularly the development of mental models in complex dynamic systems
• Keywords: Mental model development, mental model elicitation, dynamic decision-making, supply chain, thought bubbles

Research Background and Problem

Research Background

• Mental models are internal cognitive structures that people rely on when interacting with the external world, and studying them is crucial for understanding human decision-making behavior.
• Although mental models are dynamic, existing research lacks effective methods to elicit and analyze the development of mental models in complex dynamic systems.
• The authors focus specifically on complex dynamic decision-making scenarios like supply chains, studying how disruptions and information sharing influence participants' mental models.

Problems and Challenges

• Early research primarily focused on mental models in static environments, while dynamic environments are more complex but less studied.
• Reliably eliciting and analyzing the development of mental models remains a major challenge in academia.
• Existing methods for eliciting mental models are biased, such as using non-contextualized approaches.

Significance

• Understanding mental models helps optimize human-computer interaction design and improve the performance of decision support systems in dynamic environments.
• A deeper understanding of human behavior in supply chain management can enhance strategies for addressing real-world issues like drug shortages.

Related Work

• Endsley proposed the theory of "situational awareness," which serves as a framework for understanding mental models.
• Methods for eliciting mental models can include verbal descriptions, visualizations, or hybrid approaches, but lack contextualized applications.
• Existing research shows that mental models are dynamic and evolve with changes in the environment and information, with many studies focusing on improving mental models in static environments.

Proposed Solution

Methods or Solutions

• Introduction of the "Thought Bubbles" Method: A method that uses open-ended textual prompts to reveal players' mental models, enabling the dynamic tracking of mental model development.
• Integration of Contextualized Elicitation: Players provide feedback through repeated meeting scenarios in a virtual environment, reducing experimenter bias.
• Combination of Qualitative and Quantitative Analysis: Players' verbalized mental model responses are analyzed using Endsley's situational awareness theory, categorized into prediction, perception, and comprehension.

Innovations

• Fully contextualized elicitation of mental models embedded within an interactive environment, avoiding biases associated with post-experiment retrieval of long-term memory mental models.
• Use of game environments (Gamettes) as simulation tools to provide realism and interactivity, effectively capturing the expansion of mental models in dynamic decision-making tasks.

Implementation Steps and Techniques

• Develop a supply chain game scenario using the StudyCrafter platform, where players act as pharmaceutical wholesalers.
• Set up different experimental conditions, including disruptions at various points in the supply chain and varying levels of information sharing.
• Collect data on players' mental model development through open-text responses triggered by "thought bubble" prompts at periodic intervals during the game.

Research Findings

Specific Findings

• Proposed the "Thought Bubbles" method and tested its effectiveness in collecting mental model data during dynamic decision-making tasks.
• Through two experiments (involving 250 participants), demonstrated how disruption location and information sharing influence mental model development:

1. Study 1: Impact of disruption location on mental model development. The experiment found that players' perception and understanding of the supply chain environment depended on their relationship to the supply chain disruption.
2. Study 2: Impact of player behavior types on mental model development. Different behavior types (hoarders, reactors, followers) exhibited significantly different mental model development paths, with hoarders being more sensitive to future uncertainties.

Comparison with Existing Solutions

• Thought Bubbles innovatively employs a contextualized, dynamic approach to eliciting mental models, overcoming the non-contextual biases of previous methods.
• Compared to traditional interview methods, it captures a more comprehensive view of players' mental model changes and development.

Experimental Evaluation Results

• Revealed the significant dynamic nature of mental model development, with experimental variables (e.g., disruption location and degree of information sharing) significantly affecting mental model evolution.
• Validated the reliability of the "Thought Bubbles" method, showing that the elicited mental model content could explain players' decision-making behavior patterns.

Limitations and Future Directions

• Limitations:
  • The current study focuses on dynamic decision-making tasks in supply chains and does not explore applications in non-dynamic environments.
  • Verbalized elicitation of mental models may not capture all cognitive aspects.
• Future Directions:
  • Investigate whether "Thought Bubbles" can be extended to non-game environments or other interaction types (e.g., human-computer interaction design, AI mental models).
  • Expand the scale of qualitative analysis using natural language processing techniques.
  • Introduce non-verbal forms of mental model expression, such as visualizations or card-sorting tools.