Informal Embodied Auditing: Exploring Facial Emotion AI (FEAI) through Community Workshops

Paper Title

Informal Embodied Auditing: Exploring Facial Emotion AI (FEAI) through Community Workshops

Publication Info

• Topic area: Critical AI literacy and informal algorithm auditing for Emotion AI.
• Keywords: Emotion AI, Facial Emotion AI, AI literacy, informal auditing, embodied interaction, ethics, participatory design, explainable AI, sociocultural critique, tangible making.

Background and Problem

• Problem / challenge: Emotion AI (EAI), particularly Facial Emotion AI (FEAI), is controversial due to issues like scientific validity, bias, and ethical concerns. Public understanding and critical engagement with these technologies remain limited, especially among adults outside formal education.
• Significance: As EAI becomes more prevalent in applications like hiring, education, and mental health, it is crucial to foster critical AI literacy to empower individuals to understand, critique, and shape these technologies.
• Motivation and related work: Prior research has focused on AI literacy for youth and technical algorithm auditing. Informal algorithm auditing and participatory approaches have shown promise but lack emphasis on embodied and sociocultural aspects. This paper addresses these gaps by exploring how informal embodied auditing can support critical AI literacy for adults.

Solution

• Proposed approach: Explore-FEAI, an interactive FEAI model and website, combined with community workshops to enable participants to explore FEAI inputs/outputs, reflect on its mechanisms, and critically engage with its ethical and sociocultural implications.
• Novelty:
  1. Introduction of informal embodied auditing as a method for critical AI literacy through embodied and material exploration.
  2. Design of Explore-FEAI, a custom FEAI model and interactive website with explainable AI (XAI) tools for hands-on learning.
  3. Analysis of how embodied interaction and tangible making support critical engagement with FEAI.
  4. Framework for leveraging sociocultural and embodied knowledge in informal AI auditing.
• Procedure and key techniques:
  • Development of Explore-FEAI with features like Grad-CAM visualizations, image editing tools, and a history review panel.
  • Five workshops (N=30) where participants interacted with Explore-FEAI, tested varied inputs, and discussed societal and ethical implications.
  • Use of tangible materials (clay, markers, paper) for creative exploration and emotional self-reflection.
  • Reflexive thematic analysis of participant interactions and feedback.

Results

• Concrete findings:
  • Participants demonstrated growing AI literacy by reasoning about FEAI’s mechanisms, critiquing training data, and articulating sociocultural and ethical concerns.
  • Embodied and material exploration supported participants in testing inputs, reflecting on emotions, and engaging critically with FEAI.
  • Participants raised foundational critiques of FEAI’s scientific validity, including the complexity, performativity, and cultural situatedness of emotions.
• Advantage over baselines: The approach uniquely combines embodied interaction, tangible making, and informal auditing to foster critical AI literacy, addressing gaps in prior work focused on technical or youth-oriented AI literacy efforts.
• Experiments / evaluation:
  • Workshops were conducted in diverse community settings (university, gallery, vocational center, maker space).
  • Data included participant worksheets, audio/video recordings, and system interaction logs.
  • Findings were analyzed using reflexive thematic analysis to capture participants’ sensemaking and critical reflections.
• Limitations and future work:
  • Need for more scaffolding to address sensitive topics like racism and sexism in FEAI.
  • Limited exploration of regulatory futures or scenarios without EAI.
  • Future work could expand to other AI systems (e.g., voice or pose recognition) and explore broader applications of informal embodied auditing.

Summary

This study introduces informal embodied auditing as a novel approach to foster critical AI literacy, particularly for Emotion AI (EAI). Through the Explore-FEAI system and community workshops, participants engaged in embodied and material exploration to test FEAI inputs/outputs, reason about its mechanisms, and critically reflect on its ethical and sociocultural implications. Findings highlight the potential of embodied interaction and tangible making to support informal auditing and public engagement with AI technologies. This work advances methods for adult AI literacy in informal settings and proposes design considerations for participatory and educational AI initiatives.