“Good Enough!”: Flexible Goal Achievement with Margin-based Outcome Evaluation

Title of the Paper

“Good Enough!”: Flexible Goal Achievement with Margin-based Outcome Evaluation

Paper Information

• Field: Human-Computer Interaction and Behavior Change Support System Design
• Keywords: Behavior change, user experience design, interviews, service design, human-computer interaction, process control, goal setting, evaluation criteria

Research Background and Problem

• Identified Issues and Challenges:

  • Traditional goal-setting systems often rely on binary evaluation criteria, focusing solely on whether goals are achieved. This evaluation method may increase user stress and even reduce their willingness to continue trying.
  • Binary evaluation overlooks user effort, thereby diminishing motivation. For example, if a user sets a goal of 10,000 steps per day but only achieves 9,500 steps, their effort may go entirely unrecognized.
  • In the early stages of behavior change, goals may be overly challenging for users, leading to cognitive anxiety and fear of failure.

• Importance:

  • Behavior change requires sustained long-term effort, and negative emotions—such as anxiety and stress—can lead users to abandon their long-term behavior change plans.
  • To optimize behavior change support systems, it is essential to study how evaluation mechanisms can make user experiences more motivating and reduce feelings of frustration.

• Research Motivation and Related Work:

  • The authors were inspired by Statistical Process Control (SPC): in manufacturing, product quality within an acceptable margin around the target is considered satisfactory.
  • The research aims to explore how a similar concept of “flexibility” can be applied to the field of behavior change, designing a new goal evaluation standard to improve user performance and experience.

Solution

• Core Method:

  • A goal evaluation framework based on the concept of “margins” is proposed, allowing user outcomes to be considered “good enough” even if they fall short of the target, as long as they are within the margin range.
  • Introduced the “Mission Success” evaluation standard, separating it from traditional goal assessment. This multi-layered evaluation aims to psychologically alleviate user anxiety and stress caused by minor failures.

• Innovations:

  • Transferred the concept of quality deviation from Statistical Process Control to behavior change support systems, offering a more “humanized” evaluation standard.
  • Allowed separation of goals and evaluations while retaining precise goal-setting to achieve a “psychological buffer zone” effect.
  • Proposed two implementation strategies—fixed margins and random margins—to simulate more flexible goal evaluation.

• Implementation Steps and Techniques:

  • Designed two experiments:
    1. Survey Experiment (Study 1): A large-scale contextual experiment (500 participants) to study participants’ perceptions of margin-based evaluation.
    2. Field Experiment (Study 2): A 10-day field experiment (54 participants) using the FlexCoach system to analyze how margin-based evaluation affects user behavior.
  • FlexCoach system design:
    • Data Collection: Step count data collected using the Samsung Health app.
    • Feedback and Evaluation: Daily goals, margins, and mission evaluations set, providing personalized feedback.
    • Interactive Interface: Displays mission success/failure indicators, daily feedback, and weekly progress dashboards.

Research Findings

• Specific Results:

  • Margin-based evaluation significantly reduced user goal-related stress, decreased negative emotions from failure, and increased positive feedback.
  • Survey experiment results showed that users’ self-assessments were significantly higher within the margin range (Mission Success) compared to traditional binary evaluations.
  • In the field experiment, most participants still aimed to achieve the target rather than merely meeting the margin’s “minimum requirement.”

• Advantages Compared to Traditional Methods:

  • The margin-based evaluation system better motivates users to continue striving in the face of failure by reducing instances of complete failure through the “Good Enough” standard.
  • Its “humanized” characteristics enhance the social connection between users and the system, increasing long-term usage rates.

• Experiment or Evaluation Results:

  • In the survey experiment, participants’ evaluations of mission success within the margin were significantly improved, while random margin designs enhanced the interactivity and engagement of evaluations.
  • In the field experiment, although no significant impact of margins on physical activity improvement was observed, most participants acknowledged the system’s potential and support for long-term behavior change.

• Limitations and Future Directions:

  • The experiment did not find a significant improvement in users’ actual step counts due to margins, necessitating further research on their specific impact on behavior change.
  • Future studies could compare margin-based evaluation with other behavior change strategies (e.g., dual goals, adaptive goals).
  • Consider dynamic adjustments to margin sizes and adaptive designs for different types of goals and users.

Conclusion and Design Implications

• Design Implications:

  • Consider the selection of margin sizes and their relationship to goal difficulty, such as setting appropriate margins around “moderately challenging goals.”
  • Explore opportunities for margin adaptation, such as dynamically adjusting margins based on users’ daily performance.
  • Clearly separate margins and goals, using different reward mechanisms to motivate users to continuously pursue their goals.
  • Optimize the presentation of margin-based evaluation content to increase positive user feedback and long-term engagement.

• Research Significance:

  • This study is the first to propose and validate the concept of margin-based goal evaluation in behavior change support systems, providing a theoretical framework and practical recommendations for designing more flexible human-computer interaction systems.