“React”, “Command”, or “Instruct”? Teachers’ Mental Models on End-User Development

Research Background and Issues

• Identified Problems or Challenges:
  This study focuses on the mental models and challenges faced by elementary school mathematics teachers when attempting trigger-action programming (TAP) for the first time. Teachers, who generally lack programming experience, encounter difficulties in customizing the personalized functionalities of smart educational devices, particularly in understanding trigger-action logic, distinguishing between events and states, and interpreting abstract versus concrete semantic representations.

• Importance of the Issue:
  The rapid proliferation of educational technology offers interactive and personalized teaching environments, with significant potential to assist students with learning disabilities. However, teachers often struggle to effectively utilize these technological tools due to a lack of programming skills, motivation, or time. Empowering teachers through end-user development (EUD) to actively design and adjust digital device functionalities is therefore critical.

• Research Motivation and Related Work:
  EUD approaches focus on developing simple tools and strategies to help non-programmer users create and customize digital tools. Trigger-action programming (TAP) is a common EUD technique, characterized by its straightforward event-action rule design, which is widely applicable for programming smart devices. While TAP lowers the entry barrier, its limited expressiveness—especially when handling multiple trigger conditions—can lead to user errors. Existing research has shown that the specificity or abstraction of language descriptions and the accuracy of users' conceptual models significantly impact their ability to successfully complete tasks.

Solution

• Method or Solution:
  The authors investigated the mental models of elementary school mathematics teachers and analyzed the impact of language rule abstraction or specificity on teachers' ability to construct and debug trigger-action rules. The study designed two sets of language rules: "specific," directly linked to device operations, and "abstract," representing higher-level general logic rules.

• Innovative Contributions:
  Three metaphorical models of teacher-system interaction—"Command," "React," and "Instruct"—were proposed. This study is the first to explore user interaction with trigger-action rule systems from a metaphorical perspective, revealing how metaphorical models influence task accuracy and completeness.

• Implementation Steps and Techniques:

  1. Observed 28 elementary school mathematics teachers using the "Thinking-Aloud Protocol" to record their thought processes while designing and debugging rules.
  2. Utilized the SENSATION platform to provide two versions of language rule sets—specific and abstract—and studied behavioral differences under each version.
  3. Extracted teachers' metaphorical models through thematic analysis and evaluated their performance based on rule accuracy and completeness.

Research Findings

• Specific Findings:

  1. Proposed three metaphorical models:
     • Command Metaphor: Views the system as a device requiring step-by-step instructions. This model explains why teachers attempt to define redundant rules or mistakenly perceive the system as needing detailed procedural guidance.
     • React Metaphor: Partially understands trigger-action logic, perceiving the system as a mechanical tool reacting to specific events.
     • Instruct Metaphor: Teachers imagine themselves "teaching" the system to complete math exercises, particularly from the perspective of student task completion.
  2. Analyzed the impact of specific and abstract language rules on teacher behavior:
     • Specific rules (e.g., descriptions involving fixed locations and specific actions) were more readily accepted by teachers but reinforced ineffective "Command Metaphor."
     • Abstract rules (e.g., generalized expressions like "most recently inserted symbol"), though initially harder to understand, helped foster "React Metaphor" or "Instruct Metaphor."

• Advantages Compared to Existing Solutions:

  • Provided an in-depth analysis of the dynamic changes in teachers' mental models when performing technical tasks, addressing gaps in detailed cognitive studies of non-programmer end-user development models.
  • Revealed the interaction between language rule abstraction and teacher metaphorical models, offering targeted recommendations for future educational technology tool design.

• Experimental or Evaluation Results:

  • Overall, the React Metaphor and Instruct Metaphor (especially the student-centered Instruct Metaphor) performed best. The "Command Metaphor," while most prevalent, was incompatible with trigger-action logic, significantly reducing rule accuracy (average accuracy rate: 32.83%).
  • Teachers who initially used more abstract rules demonstrated more significant learning progress, suggesting that starting with more challenging rule frameworks can promote the formation of correct metaphorical models.

• Limitations and Future Directions:

  • The sample size was small, and participants had a homogeneous background (primarily Italian elementary school female teachers with no programming experience), which may limit the generalizability of the results.
  • The design of platform language rules may have influenced teacher understanding; future research could explore optimizing abstract rule descriptions.
  • Future studies could extend to teachers in other domains and more diverse EUD task designs to verify the broad applicability of metaphorical models.

This study provides a new perspective on end-user development behavior in educational technology and offers specific recommendations for designing more intuitive and effective programming tools.