'It Is Not Always Discovery Time': Four Pragmatic Approaches in Designing AI Systems

Document Title

‘It Is Not Always Discovery Time’: Four Pragmatic Approaches in Designing AI Systems

Document Information

• Subject Area: Human-Computer Interaction and AI Design
• Keywords: Artificial Intelligence, Design Process, Data Handling, Design Methods, Human-Computer Interaction

Research Background and Issues

• Problems and Challenges: As artificial intelligence (AI) systems increasingly integrate into everyday technological applications, there remains insufficient understanding of how designers collaborate with AI. Many AI-related design processes are fraught with uncertainties in practice, particularly in collaboration with data scientists and developers, where communication barriers often arise. Additionally, aligning AI functionalities with user needs is a significant challenge.

• Significance: The design of user-facing AI products directly determines the effectiveness and user experience of AI systems. Researching how to optimize the design process of AI products not only enhances product value but also provides guidance for cultivating future AI design talent.

• Research Motivation and Related Work:

  • Previous studies have primarily focused on the practices of experienced designers, overlooking how designers with varying levels of experience participate in AI projects.
  • New challenges in AI design include understanding AI's complex capabilities, the workload of data handling, and the coordination difficulties between design and development teams.
  • This study aims to expand the understanding of the AI design field by investigating how designers from diverse backgrounds and experience levels adjust their design processes to meet AI requirements.

Solutions

• Methods and Solutions:

  • Conducted semi-structured interviews with 20 designers to explore their practical practices and process adjustments in AI design projects.
  • Proposed four categorized design approaches: a priori (prior methods), post-hoc (post methods), model-centric (model-centered methods), and competence-centric (competence-centered methods).

• Innovations:

  • Linked the design process to Mueller et al.'s five-stage model of human intervention in data science (discovery, capture, management, design, creation) to better analyze intervention points and challenges in AI design.
  • Provided a method to re-examine design processes from the perspective of team dynamics and collaboration skills, offering inspiration for diverse design teams.

• Implementation Steps and Techniques:

  1. Designed a pre-interview questionnaire with background information to collect participants' project contexts and demographic data.
  2. Conducted in-depth interviews to explore participants' specific design practices, interactions with data, team collaborations, and encountered difficulties.
  3. Performed thematic analysis on the interview data, extracting four main themes: data and models, processes, teams, and translation.

Research Findings

• Specific Findings:

  • Identified four methods in the AI design field:
    • a priori: Treats the AI model as a completed static module and designs the interface around it.
    • post-hoc: Completes UI design first, then has the AI team implement the corresponding functionalities.
    • model-centric: Drives the entire design and development process through the data model, emphasizing synchronized iteration between the model and design.
    • competence-centric: Emphasizes division of labor between design and development teams but ensures process alignment through continuous synchronization.
  • Identified major challenges for AI design teams, including communication difficulties with developers and data scientists, issues arising from data quality discrepancies, and decision-making challenges in aligning user needs with AI capabilities.

• Comparison with Existing Methods:

  • The findings differ from previous studies by emphasizing the diversity in interaction levels between design and technology, describing a gradual progression from low technical involvement (a priori and post-hoc methods) to deep technical collaboration (model-centric and competence-centric methods).
  • Advocates for replacing the "idealized" descriptions of design processes with more pragmatic approaches.

• Experiment and Evaluation Results:

  • The success of AI design teams relies on multidisciplinary collaboration and knowledge sharing.
  • Designers require a certain level of foundational AI knowledge to communicate more effectively with technical teams and advance prototype development and data validation work.

• Limitations and Future Directions:

  • Limitations:
    1. Although the study sample included diverse participants, it did not encompass design teams from different cultural backgrounds and organizational types globally.
    2. There is room for deeper analysis of the roles within the entire team.
    3. The time slices of the design process may be more complex and were not fully captured.
  • Future Directions:
    1. Explore how to systematically integrate AI into the design process.
    2. Develop instructional models for interaction design education to enhance design expertise tailored to AI technologies.
    3. Investigate how organizational scale and cultural background influence the selection and optimization of AI design processes.

By clearly categorizing and comparatively analyzing design approaches, this study provides critical perspectives for understanding and improving AI design practices while laying a theoretical foundation for further exploration of education and training in the AI design field.