Perceived Empathy of Technology Scale (PETS): Measuring Empathy of Systems Toward the User

Title of the Paper

Perceived Empathy of Technology Scale (PETS): Measuring Empathy of Systems Toward the User.

Paper Information

• Subject Area: Human-Computer Interaction (HCI), Affective Computing
• Keywords: Human-Computer Interaction, Affective Computing, Technological Empathy, Scale Development, User Experience, Reliability Validation, Confirmatory Factor Analysis, Trust

Research Background and Problem

• Problem or Challenge: The authors identified that while existing methods can typically measure empathy between humans, there is a lack of reliable and validated scales for measuring perceived empathy in interactive systems. Many studies currently use scales designed for measuring human empathy, which are often not sufficiently adapted to system contexts, leading to validity issues when assessing the perceived empathy of systems.
• Significance: With advancements in affective computing and artificial intelligence, interactive systems such as conversational agents and social robots have demonstrated the ability to express some form of "empathy" toward users. Measuring the perceived empathy of these systems can help evaluate their effectiveness and support the development of more emotionally intelligent technologies.
• Research Motivation: The authors aim to address the aforementioned gaps by developing a dedicated scale to quantify system empathy from the user's perspective, ensuring its generalizability across different applications.

Solution

• Method or Solution:
  • The authors proposed and developed a new scale called the Perceived Empathy of Technology Scale (PETS).
  • PETS is divided into two major factors: Emotional Responsiveness (PETS-ER) and Understanding and Trust (PETS-UT).
• Innovations:
  • The scale was developed from scratch rather than simply adapting existing human empathy scales.
  • The development of PETS strictly followed the multi-phase systematic approach to scale development proposed by Boateng et al., involving expert interviews, item generation, item validation, factor analysis, validation, and reliability testing.
  • The authors designed 22 scenarios with varying levels of empathy specifically for interactive systems to ensure the scale's broad applicability.
• Implementation Steps:
  1. Item Generation: Data from semi-structured interviews with 18 experts were coded to generate an initial pool of 100 items.
  2. Content Validation: Eight domain experts rated the relevance of the items, and the Content Validity Index (CVI) was used to filter the items, reducing the pool to 38 items.
  3. Item Testing and Reduction: A total of 324 participants rated the 38 items. Exploratory Factor Analysis (EFA) was conducted, further reducing the items to 10.
  4. Confirmatory Factor Analysis (CFA): Two independent samples were used for testing, ultimately confirming the two-factor structure of the 10 items.
  5. Validation and Reliability Testing: This included internal consistency (Cronbach's α), split-half reliability, test-retest reliability, discriminant validity, and convergent validity.

Research Findings

• Specific Outcomes:
  • A 10-item, two-factor scale (PETS) was developed, with PETS-ER comprising 6 items and PETS-UT comprising 4 items.
  • The scale demonstrated excellent internal consistency (α=0.96) and significant test-retest reliability (ICC=0.94).
  • The scale effectively distinguished between "high-empathy" and "low-empathy" interactive system scenarios.
• Advantages Compared to Existing Solutions:
  • PETS addresses the validity issues of existing scales by specifically measuring technological empathy from the user's perspective.
  • Its broad applicability was validated through tests across 22 different scenarios, including evaluations of conversational agents, social robots, and functional applications.
• Experimental or Evaluation Results:
  • In experimental scenarios, user-perceived empathy scores for systems with empathy were significantly higher than those for systems without empathy.
  • The two-factor structure of PETS was well-validated across multiple samples.
  • In testing convergent validity, PETS-ER was strongly correlated with emotional understanding, while PETS-UT was more aligned with assessing user perceptions of system trust and goal understanding.
• Limitations and Future Directions:
  1. Limitations:
     • The study was based on video and audio rather than real interactive systems, so performance in real-world applications may differ.
     • The current scale only includes positively framed items, potentially introducing a positive response bias.
     • Negative or potentially malicious dimensions of empathy were not included.
  2. Future Directions:
     • Further validation of the scale in real systems and dynamic interaction contexts.
     • Adding items that reflect potential malicious uses of empathy.
     • Exploring the scale's effectiveness in predicting user behaviors (e.g., trust-building, behavior change).
     • Applying the scale to more practical scenarios (e.g., therapeutic robots, educational assistants).

Conclusion

PETS provides a comprehensive, reliable, and generalizable tool for quantifying the perceived empathy of technological systems, supporting advancements in the field of interactive technology and contributing significantly to the development of emotionally intelligent systems. This study also offers a clear framework and valuable insights for designing, validating, and optimizing scales related to affective computing.