Sensemaking With/About AI: Unpacking Design Professionals’ Data Sensemaking Styles in a High-Stakes Industrial Context

Paper Title

Sensemaking With/About AI: Unpacking Design Professionals’ Data Sensemaking Styles in a High-Stakes Industrial Context

Publication Info

• Topic area: AI-supported data sensemaking in high-stakes UX design contexts.
• Keywords: AI-supported tools, data sensemaking, UX design, high-stakes design, truck interaction design, AI literacy, human-AI collaboration, sensemaking strategies, explainability, interactive prototypes.

Background and Problem

• Problem / challenge: Limited research exists on how designers engage with AI for data sensemaking in real-world, high-stakes contexts like vehicle design. Existing tools often fail to address the uncertainties and complexities of AI outputs and their integration into professional workflows.
• Significance: Understanding and improving AI-supported sensemaking is critical in high-stakes domains where safety, regulatory, and accountability demands require defensible, data-driven design decisions.
• Motivation and related work: Prior work has explored AI's role in automating tasks and generating insights but has not adequately addressed how designers interpret and validate AI outputs in real-world, high-stakes settings. This study builds on existing theories of sensemaking and AI-human collaboration to address this gap.

Solution

• Proposed approach: Co-design and evaluation of an AI-supported data sensemaking prototype tailored to truck interaction design, a high-stakes industrial context.
• Novelty:
  1. Characterization of three sensemaking styles (expert heuristics, data-grounded analysis, dynamic sensemaking) in AI-supported design.
  2. Identification of a bidirectional relationship between AI and data in sensemaking processes.
  3. Design implications for tools that support diverse sensemaking approaches, foster AI literacy, and provide interactive and explainable features.
• Procedure and key techniques:
  1. Phase 1: Prototype Co-Design: Observations, workshops, and iterative design of a high-fidelity prototype with features like AI-generated scenario summaries, explainability elements, and interactive controls.
  2. Phase 2: Prototype Evaluation: Think-aloud sessions with 17 participants from the truck industry to explore how they interpreted AI-generated insights and engaged with the tool's features.

Results

• Concrete findings:
  • Identified a bidirectional relationship where AI helps interpret data, and data helps validate AI outputs.
  • Three sensemaking styles emerged: expert heuristics (relying on domain expertise), data-grounded analysis (frequent cross-checking with data), and dynamic sensemaking (flexible use of expertise, data, and interaction).
  • Participants highlighted the importance of explainability, granular data access, and interactive control for effective sensemaking.
• Advantage over baselines: The prototype enabled designers to explore user scenarios more effectively, contextualize AI outputs, and address uncertainties in high-stakes design decisions, surpassing traditional static data visualization tools.
• Experiments / evaluation:
  • Participants: 17 professionals with an average of 5+ years of experience in truck design.
  • Tasks: Exploring AI-generated user scenarios, validating outputs, and interacting with features like clustering, timelines, and anomaly detection.
  • Metrics: Qualitative thematic analysis of sensemaking strategies, trust levels, and perceived control.
• Limitations and future work:
  • Findings are based on a prototype, not a fully functional system; future longitudinal studies are needed to observe evolving sensemaking strategies.
  • Limited to experienced truck designers; future work should explore diverse expertise levels and other high-stakes domains.
  • Further research is needed to refine AI literacy cultivation and interactive features like traceability and steerability.

Summary

This study investigates how UX designers engage with AI to make sense of interaction data in high-stakes contexts, using truck interaction design as a case study. It identifies a bidirectional relationship between AI and data in sensemaking and categorizes three distinct sensemaking styles. The co-designed prototype demonstrated the importance of interactive and explainable features in supporting diverse sensemaking strategies and fostering AI literacy. These findings contribute actionable insights for designing AI-supported tools that empower critical and dynamic engagement with data in professional, high-stakes settings.