AffectiveSpotlight: Facilitating the Communication of Affective Responses from Audience Members during Online Presentations

Title of the Paper

AffectiveSpotlight: Facilitating the Communication of Affective Responses from Audience Members during Online Presentations

Paper Information

• Domain: Human-Computer Interaction and Affective Computing
• Keywords: Affective computing, public speaking, intelligent user interface, video conferencing, user experience, human-computer interaction, emotional response, facial expression analysis, live interaction, user feedback

Research Background and Problem Statement

• Problems or Challenges:

  • Limited online audience feedback: Existing video conferencing platforms (e.g., Microsoft Teams) fail to adequately capture audience reactions during online presentations, restricting the speaker's ability to adapt dynamically to audience emotions.
  • Expression pressure: Public speaking can lead to speaker anxiety due to the lack of audience interaction, especially when facing neutral or passive audiences.
  • Technological limitations: Current video conferencing applications lack effective mechanisms to capture non-verbal feedback (e.g., nodding, smiling, facial expressions), negatively impacting the user experience during remote work.

• Significance:

  • Understanding and dynamically adapting to audience feedback is a key factor in improving the quality of online presentations, helping to alleviate speaker anxiety and enhance the public speaking experience.
  • Addressing this issue is particularly important in the context of large-scale remote work, as it can effectively bridge the gap between online and offline interaction experiences.

• Research Motivation and Related Work:

  • Inspired by the spotlight technique in theater and cinematography, the authors aim to leverage real-time affective computing technology to enhance the experience of online presenters and audience interaction.
  • Existing research primarily focuses on capturing audience responses in face-to-face scenarios, with limited exploration of emotion-driven feedback systems in online contexts.

Solution

• Method or Solution:

  • Proposing “AffectiveSpotlight,” a real-time video analysis bot integrated with Microsoft Teams. The system utilizes affective computing technology to analyze audience facial expressions and head movements, dynamically spotlighting the most responsive audience members.
  • The system extracts cognitive states (e.g., confusion, engagement) and audience behaviors (e.g., nodding, shaking heads) through real-time video analysis, showcasing the most relevant audience members to the presenter.

• Innovative Contributions:

  1. Combining the theatrical “spotlight” concept with affective computing to dynamically capture audience reactions and optimize feedback reception for presenters.
  2. Employing implicit analysis methods to eliminate the interference of traditional explicit feedback methods (e.g., surveys, polls), reducing cognitive load.
  3. The system avoids directly labeling or inferring emotions, instead preserving ambiguity through video representation, allowing presenters to interpret audience reactions based on context.

• Implementation Steps and Key Technologies:

  1. Audience Feature Capture: Utilizing Microsoft Face API for facial and head keypoint detection, combined with convolutional neural networks to analyze emotional states (e.g., happiness, sadness) and actions (e.g., nodding).
  2. Weighted Scoring Mechanism: Calculating emotion-weighted scores for audience video frames, selecting the highest-scoring audience member every 15 seconds as the spotlight target.
  3. Integration with Microsoft Teams: Developing a Teams bot to retrieve real-time video input and display spotlight video streams on the presenter’s screen.

Research Outcomes

• Specific Results:

  • Comparative studies show that AffectiveSpotlight significantly enhances presenters’ audience awareness (including connection and adaptability to audience reactions), improving platform satisfaction and usability compared to random audience selection or default UI conditions.
  • Presenters using AffectiveSpotlight reported self-assessments closer to audience evaluations, indicating the system’s effectiveness in establishing better feedback mechanisms.
  • The system notably reduced presenter anxiety and extended presentation duration.

• Advantages over Existing Solutions:

  • Compared to traditional methods relying on aggregated data, AffectiveSpotlight intuitively displays audience reactions via video streams, reducing cognitive load for presenters.
  • The system addresses the lack of interactive feedback in random audience selection or default UI setups.

• Experimental and Evaluation Results:

  • Testing with 14 groups and 117 participants demonstrated the system’s positive impact on meeting performance and improving online presentation experiences.
  • Visual analysis accuracy benefited from controlled experimental conditions (e.g., camera positioning and lighting), though challenges remain in achieving system generalization.

• Limitations and Future Directions:

  • Limitations:
    • The controlled experimental environment may differ from real-world usage scenarios (e.g., meeting scale, presentation content).
    • The audience group primarily consisted of employees from technology companies, limiting generalizability to diverse user backgrounds.
    • The spotlight mechanism may overly emphasize certain facial features under specific conditions, requiring enhanced flexibility.
  • Future Directions:
    • Explore more dynamic and adaptive window selection mechanisms, allowing users to customize spotlight behavior.
    • Extend sensing technologies (e.g., microphones, eye-tracking) to support richer emotion detection.
    • Optimize privacy and transparency mechanisms to improve user trust and acceptance of the system.