Observe, Ask, Intervene: Designing AI Agents for More Inclusive Meetings

Research Background and Issues

• Issues and Challenges:
  The authors observed that achieving inclusion in video conferencing is a significant challenge. Traditionally, meeting inclusivity often relies on the behavior of meeting leaders or co-hosts, placing immense pressure on them. Without proper training or resources, many teams struggle to manage inclusivity effectively.

• Importance:
  Inclusivity is critical in organizations as it enhances employee satisfaction, job performance, and creativity. In the context of video conferencing, inclusivity issues are particularly pronounced, as technological and social barriers may exacerbate inequalities during meetings.

• Research Motivation and Related Work:
  While existing research has explored how technology can support group work, most solutions fail to effectively address the social dynamics of meetings. Previous studies have demonstrated that technology can improve meeting experiences through information delivery and behavioral prompts, but there remains a design gap in scalable inclusivity interventions.

Solution

• Method and Framework:
  The authors propose the "Observe, Ask, Intervene" (OAI) framework, which consists of the following three stages:

  1. Observe: Record behavioral patterns during meetings, such as the speaking time of each participant.
  2. Ask: Pose questions to participants to confirm identified issues and collect feedback from them.
  3. Intervene: Based on the feedback, provide private suggestions and visualized data support to the host and other key participants (e.g., those who dominate the conversation).

• Innovations:

  1. The OAI framework reduces reliance on AI inference accuracy by incorporating explicit user input.
  2. Introduces the new role of an AI "virtual co-host" to alleviate discomfort caused by direct feedback in social interactions.
  3. Emphasizes social comfort—for instance, by avoiding public criticism through private feedback and allowing users to retain partial control over system behavior.

• Implementation Steps and Key Technologies:

  • The system observes speaking time in real-time during meetings, triggering rule-based feedback loops.
  • The AI system employs non-intrusive design in user interactions, such as sending suggestions and charts via private messages.
  • Although prototype-based, the design is compatible with more advanced AI technologies like deep inference or sentiment analysis.

Research Outcomes

• Specific Outcomes:

  1. The authors developed and tested a prototype of a "virtual co-host" based on the OAI framework.
  2. User experiments showed that introducing a virtual co-host improved subjective evaluations of meeting quality.
  3. The OAI framework significantly enhanced users' sense of agency and trust, though it had limited success in fundamentally altering behavior.

• Relative Advantages:
  Compared to existing systems, the main advantages of the OAI approach lie in its reliance on user feedback and its use of private feedback to avoid direct confrontation or a "culture of shaming."

• Experimental Results:

  • The experiment involved 68 participants and 18 in-depth interviews. Key findings include:
    • Ask Phase: Positive interactions during this phase led to higher user trust.
    • Intervene Phase: While private feedback was widely accepted, it failed to significantly alter user behavior, particularly among those who dominated conversations.
  • Survey data suggested that the presence of the virtual co-host may have improved users' subjective evaluations of meeting quality through psychological cues.

• Limitations and Future Directions:

  1. Limitations:
     • The experiment was conducted in a highly controlled environment, which may not fully replicate real-world corporate scenarios.
     • The system failed to exert significant social pressure on users who disregarded feedback.
     • Most data were collected from U.S.-based academic and industrial settings, potentially limiting its applicability to other cultural or linguistic contexts.
  2. Future Directions:
     • Conduct more complex, long-term evaluations and tests in real-world environments (e.g., various meeting types and team sizes).
     • Explore more effective integration of social pressure and AI tools (e.g., introducing semi-public feedback mechanisms).
     • Test the OAI framework in other cultural contexts to assess its applicability.

Conclusion

This paper contributes the OAI framework and validates its potential value through experiments. Although its current intervention methods have limited success in changing behavior, it provides important insights for designing more effective virtual meeting tools. Additionally, the study highlights the challenges and opportunities in designing AI systems to promote social behavior change, pointing the way for future research.