The Framework of the Lived Experience of Metrics: Understanding the Purposes and Activities of Self-Tracking Metrics

Research Background and Issues

What problems or challenges did the authors identify?

• Most research in Personal Informatics (PI) focuses on users' overall self-tracking experiences or achieving predefined goals, with limited exploration into how users understand the complex metrics generated by trackers.
• Commercial systems continuously introduce new "derived metrics" (e.g., stress scores or body battery energy levels), which users often struggle to comprehend and utilize effectively.
• Misunderstanding certain metrics may lead to improper usage, thereby affecting the reliability of results and the efficiency of health management.

Why is this issue important?

• Studying how users select and interpret these metrics can provide design guidelines for creating more effective tracking systems in the future.
• Understanding user interactions with complex metrics is crucial for improving health-related technological tools, especially in measuring psychological and physiological health.

Research Motivation and Related Work

• Previous research models, such as the Stage-Based Model of Personal Informatics and the Lived Informatics Model, describe user processes but lack detailed discussions on how users select and understand metrics.
• The authors aim to investigate the differences in user experiences between low-level and high-level metrics (direct measurements vs. derived metrics) to address this knowledge gap.

Solutions

What methods or solutions did the authors propose?

• The authors introduced the "Lived Experience of Metrics Framework" (LivEM), which categorizes users' purposes (why) and activities (how) in using metrics.
• Using semi-structured interviews (25 participants) and a confirmatory online survey (63 participants), they explored how users utilize and adapt to these metrics.
• They defined two core components of the framework: Purposes and Activities, which respectively describe why users choose metrics and how they interact with them.

What is innovative about this solution?

• Innovation 1: The LivEM framework analyzes users' metric selection and adaptation behaviors across two dimensions (purposes and activities), emphasizing the specific usage processes of metrics, which differs from existing models.
• Innovation 2: The framework reveals the dynamic nature of metric usage and connects it to the stages of users' PI journeys, offering a more comprehensive understanding of metric significance and user contexts.
• Innovation 3: It highlights the complexity of user-metric interactions, such as reselecting metrics or redefining goals based on specific situations.

Implementation Steps and Key Techniques

1. Interview Phase: Semi-structured interviews explored how users select metrics and their purposes for usage, including the impact of metrics on behavior.
2. Data Analysis: Qualitative analysis of interview recordings using ATLAS.ti software to extract themes and construct the framework.
3. Validation Survey: An online questionnaire validated interview findings, examining the frequency and types of metrics used by users.
4. Framework Design: Based on interview and survey results, the LivEM framework was quantified and constructed, emphasizing the interaction between the two dimensions.

Research Outcomes

What specific outcomes were achieved?

• Framework Structure: The LivEM framework identified two core dimensions:
  • Purposes: Monitoring one's state, goal-setting, self-comparison, social comparison, curiosity.
  • Activities: Interpreting, contextualizing, re-selecting.
• Behavioral Patterns: Users tend to engage in activities driven by metric purposes (e.g., interpreting body status through "monitoring"), and these behaviors are closely tied to the stages of their PI journeys.

How does it compare to existing solutions?

• Unlike traditional models that adopt a macro-level perspective, the LivEM framework focuses on specific user scenarios and behavioral details in metric usage.
• It provides a methodology to understand how users dynamically select and adapt to complex metrics (e.g., "body battery" or "stress scores").
• It clarifies that metric design should consider users' personalized needs, reference point design, and the balance between metric complexity and user comprehension.

What were the experimental or evaluation results?

• Interviews and survey data revealed that most users engage with metrics for multiple purposes and activities.
• Key Findings:
  • Certain metrics (e.g., step count, sleep) are suitable for diverse purposes and activities, while more complex metrics (e.g., stress scores) are typically limited to specific purposes.
  • Activities such as "contextualizing" are more universally applicable across metrics, whereas purposes are more specific.

Limitations and Future Directions

• Limitations:

  • Data collection was concentrated in the EU region, resulting in a relatively homogeneous cultural background that may not fully reflect global user behaviors.
  • The study primarily focused on health and fitness domains, leaving the applicability of the framework to other PI tools (e.g., education or work performance tracking) unverified.

• Future Directions:

  • Further research into the applicability of LivEM in other PI domains (e.g., education, productivity management).
  • Investigate the impact of broader cultural contexts on metric selection and interpretation.
  • Explore more effective ways to design metrics in complex systems, enabling users to set more reasonable usage paths based on the framework's purposes and activities.

This study and the proposed framework are expected to provide significant insights for the future design of personal informatics tools, helping users enhance their health management capabilities while avoiding negative effects caused by misunderstanding complex metrics.