Explorable Explainable AI: Improving AI Understanding for Community Health Workers in India

Document Title

Explorable Explainable AI: Improving AI Understanding for Community Health Workers in India

Document Information

• Subject Area: Explainable Artificial Intelligence (XAI), Human-Computer Interaction (HCI), Global South Development and Healthcare Technology
• Keywords: Explainable AI, Human-Computer Interaction, Community Health Workers, Global South, Learning Tools, Digital Literacy, User Research

Research Background and Problem

• Identified Problems or Challenges:

  • AI technologies are being used to assist community health workers (CHWs) in rural India in diagnosing issues like child malnutrition, but little is known about how these technologies are understood by CHWs with low digital literacy.
  • The complexity and lack of transparency in AI technologies may hinder their safe and effective application in resource-constrained environments.
  • Power imbalances and socioeconomic pressures in CHWs' work environments further exacerbate barriers to technology integration.

• Significance of the Problem:

  • Child malnutrition accounts for 45% of deaths among children under five globally, making it a critical issue in the United Nations Sustainable Development Goals.
  • CHWs provide essential health services in remote areas but face challenges such as insufficient training, lack of technical support, resource shortages, and low wages, making the application of AI technologies crucial.

• Research Motivation and Related Work:

  • This study aims to explore how designing "Explorable Explanations" can help people better understand AI, foster trust, and encourage responsible use of these tools.
  • It expands the research in Explainable AI by shifting the focus from the Global North to the Global South, emphasizing the feasibility and audience-specific needs of tools in resource-limited settings.

Solution

• Proposed Method or Solution:

  • The authors designed a user interface called "Explorable Explanations," which integrates interactive visual cues and explanatory text to facilitate active learning for community health workers.
  • The study evaluates this mechanism using a "Design Probe" approach to investigate how it helps participants understand AI without requiring extensive technical or digital backgrounds.

• Innovative Aspects:

  • Explorable Explanations are based on the concept of "play," allowing users to test AI predictions and model behavior through real-time adjustments and interactions. This user-centered approach reduces the cognitive load required to understand AI.
  • The study is the first to explore in detail how to help digital novices understand AI and proposes design principles tailored to low cultural and social contexts.

• Implementation Steps and Key Technologies:

  • Developed a Figma prototype tool simulating an AI-driven child malnutrition prediction application, incorporating Explorable Explanations.
  • Included the following interactive modules:
    1. Feature Information Mode: Displays definitions and measurement methods for each nutritional indicator.
    2. Edit Measurement Module: Allows users to adjust children's measurement parameters and observe changes in predictions.
    3. Feature Importance Interface: Explains which indicators influence child health classifications.
    4. Comparison Screen: Compares prediction results for different children's nutritional statuses.

Research Outcomes

• Specific Findings:

  • Compared to systems without explanation features, the inclusion of Explorable Explanations significantly enhanced CHWs' understanding of how AI works. Many participants recognized that AI predictions are based on statistical models rather than being entirely consistent with their manual assessments.
  • Explorable Explanations not only improved CHWs' ability to critique AI predictions but also encouraged them to approach AI predictions with greater caution and critical thinking.

• Advantages Over Existing Solutions:

  • Interactive design demonstrated a clear advantage over current generic explainability methods (e.g., LIME or SHAP) in improving user understanding and learning capabilities.
  • In economically and socially resource-constrained regions, interactive design lowered the barriers to understanding and applying technology.

• Specific Experimental Results:

  • Among 30 participants, as they engaged with the design probe, folk theories about AI were revised, and some participants developed systematic understandings, strengthening their independent judgment and ability to challenge incorrect predictions.
  • However, participants generally struggled to understand the "Confidence Slider," confusing it with the severity of the prediction.

• Limitations and Future Directions:

  • The design probe could not fully simulate real-world AI application scenarios; future work should develop more complete functional prototypes.
  • The current study lacks broad quantitative validation; future research could use controlled experiments to evaluate the impact of different explanation types on participants' understanding.
  • Investigate how participatory design methods can incorporate CHWs' needs to improve AI technology development and deployment.

Conclusion

The authors highlight the significant potential of Explorable Explanations in enhancing CHWs' understanding of AI but emphasize the need to account for the social, environmental, and cultural constraints of real-world applications. Future research should focus on transitioning from AI transparency to explanation methods that support play, learning, and social structures, fostering the sustainable development of AI tools with a deep socio-technical perspective.