Farsight: Fostering Responsible AI Awareness During AI Application Prototyping

Document Title

Farsight: Fostering Responsible AI Awareness During AI Application Prototyping

Document Information

• Subject Area: Artificial Intelligence, Responsible AI Development, Tool Design, and Human-Computer Interaction
• Keywords: Responsible AI, Human-AI Collaboration, Large Language Models, Prototype Development, Social Harm Assessment

Research Background and Problem

• Problem and Challenges: With the rapid growth of AI, large language models (LLMs) and prompt tools have made AI application prototyping more accessible. However, this stage often fails to identify potential social harms, especially for non-technical users such as designers and lawyers. Existing tools primarily target machine learning experts and cannot effectively assist diverse user groups in predicting or mitigating these risks.
• Importance: Preventing and reducing harm in the early stages of application development is a crucial step to avoid the negative social impacts of technology. Addressing this issue contributes to enhancing the responsibility and safety of AI.
• Research Motivation: Establishing support mechanisms during the early design phase of generative AI tools to raise developers' awareness of social harm while integrating related frameworks, such as harm scenario modeling.

Solution

• Method and Tool: An interactive tool named Farsight is proposed, integrated into existing AI prototyping environments (e.g., Google AI Studio), to help users identify potential social harms in LLM application development.
  • The tool uses embedded similarity analysis to prompt users with AI incident reports related to their inputs.
  • It provides potential use cases, affected stakeholders, and possible harms generated by LLMs.
  • Includes an interactive interface (e.g., node tree) allowing users to edit, add, delete harms, or generate content.
• Innovation: The tool is designed not only for technical personnel but also for users from diverse backgrounds, enabling harm prediction during the development phase through human-AI collaboration. It extends the concept of responsible AI to non-technical groups using a progressive disclosure design to minimize user disruption.
• Implementation Steps and Technologies:
  1. Real-Time Alerts: Embedded similarity analysis displays relevant AI incidents and risks.
  2. Sidebar Content Presentation: Provides clickable expandable news events and LLM-generated use cases with harm classifications.
  3. Harm Envisioner Module: Interactive node tree for users to visualize and edit generated content.
  4. Technical Architecture: Supported by TensorFlow.js, D3.js, and other technologies for machine learning and data visualization.
  5. Open-Source Integration: Can be integrated into any web environment for further research and development.

Research Outcomes

• Specific Outcomes:
  • The Farsight tool significantly enhances users' ability to independently predict social harms.
  • Helps users expand their focus to include indirect stakeholders and cascading harms.
  • Compared to existing responsible AI tools, it notably improves users' ability to construct harm scenarios autonomously.
• Experiment and Evaluation Results:
  • An evaluation study with 42 AI prototyping users showed that using Farsight significantly increased the number of identified harms.
  • Users expressed high approval of the tool's design, finding it more useful and user-friendly than existing resources.
  • Quantitative experiments demonstrated that versions with higher user engagement achieved better performance.
• Advantages and Limitations:
  • Advantages: The tool excels in generating non-traditional user scenarios compared to existing tools and encourages users to actively reflect on AI harms.
  • Limitations:
    • The quality of generated content may not fully meet user expectations in certain cases.
    • The tool does not provide actionable harm mitigation recommendations.
• Future Directions:
  • Further exploration of providing substantive and specific harm mitigation recommendations during the prototyping phase.
  • Conducting long-term studies to measure the lasting impact of users' responsible AI awareness.
  • Improving the quality and personalization of generated content for diverse user backgrounds.