Embodying the Algorithm: Exploring Relationships with Large Language Models Through Artistic Performance

Title of the Paper

Embodying the Algorithm: Exploring Relationships with Large Language Models Through Artistic Performance

Paper Information

• Subject Area: Human-Computer Interaction (HCI), Algorithm Studies, Large Language Models (LLMs), and Artistic Performance
• Keywords: embodiment, human-computer interaction, GPT-3, algorithm, performance art, sense-making, AI collaboration, physicality, artistic research, agency

Research Background and Problems

• Issues or Challenges:

  • Despite the widespread application of algorithmic systems in human life, there is limited research focusing on the "materiality" and "physicality" of human-algorithm interaction processes.
  • Current algorithms (especially LLMs like GPT-3) often generate instructions that overlook human physical limitations, directly or indirectly leading to failures in "collaboration" with humans.
  • In various contexts, humans tend to anthropomorphize algorithms, attributing intentions and meanings to them, even though algorithms lack such capabilities.
  • Existing studies on algorithmic behavior primarily focus on technical design aspects, with insufficient exploration of their impact on physical and sensory experiences.

• Significance of the Research:

  • Modern algorithms are deeply embedded in social life, such as content recommendation, advertising, voice assistants, etc., shaping human behaviors and lifestyles.
  • Exploring the interaction and physicality between humans and algorithms can provide a more comprehensive understanding of the social effects of algorithms, particularly the possibilities and limitations of human-algorithm collaboration.
  • The study can reveal the inequalities and risks arising from algorithms neglecting human physicality and complex social contexts.

• Research Motivation:

  • The study proposes a novel approach to interacting with algorithms through artistic performance, aiming to uncover how these algorithms influence human experiences and how humans respond through their embodied preconceptions.

Solution

• Methods or Solutions:

  • The authors conducted an experimental project called "Embodying the Algorithm", combining artistic research with human-computer interaction analysis. Specifically:
    • Artistic Performance: Professional performance artists were invited to engage in endurance performances based on GPT-3-generated instructions.
    • Embodiment Experiments: By having performance artists "execute" the generated texts physically, the study explored the direct impact of algorithm-generated instructions on the body and captured how humans reflect and interpret these instructions through their physicality.
    • Multimodal Relationship Analysis: The interactions between artists and GPT-3 were categorized into three possible relational modes (relationships with the algorithm rather than collaboration).

• Innovative Aspects:

  1. Designing extreme algorithm-generated instructions to interact with human physical boundaries, breaking the traditional "abstract" limitations of algorithm studies.
  2. Using endurance performance as a method to explore the tension and meaning-making between the human body and algorithms.
  3. Deep analysis of the infeasibility of traditional "collaboration" between humans and algorithms.

• Implementation Steps:

  • Instruction Generation: Using GPT-3 to generate a set of performance instructions (via "artistic performance input text prompts").
  • Experimental Design: Five performers selected instructions, interpreted them, and recorded their performances.
  • Qualitative Interviews: The study collected data through in-depth interviews and performance observations to analyze how participating artists interpreted and responded to GPT-3.
  • Data Analysis: Patterns of interaction were summarized and analyzed based on interviews and performance results.

• Key Technologies:

  • GPT-3 as a language generation model to produce initial performance instructions.
  • Artistic research methods as tools for knowledge production supporting the experimental framework.

Research Findings

• Specific Findings:

  1. Three Modes of Human-Algorithm Relationships:

     • Agonistic Mode: When instructions ignored physical limitations, performers resisted and reshaped the instructions, expressing defiance.
     • Perfunctory Mode: Performers mechanically completed tasks without emotional engagement or speculation about the algorithm's intentions.
     • Agreeable Mode: Performers occasionally exhibited "pleasurable" compliance with the algorithm, though this was often a misjudgment, assuming the algorithm could understand or respond.

  2. Key Methods for Perceiving Generated Patterns:

     • Reflexivity, interpretation, and personification were employed to sense and interpret algorithmic text results, despite humans persistently attributing "intentions" to the algorithm.

  3. Tensions Between Algorithms and Physical Limitations:

     • GPT-3-generated instructions showed a lack of concern for the executing subject, such as ignoring physiological fatigue or even life-threatening scenarios—e.g., instructions involving resurrection after death.

• Advantages Compared to Existing Research:

  • The study offers a unique perspective on the direct impact of algorithms on the human body, addressing the shortcomings of overly "abstract" algorithm studies.
  • It explores conflicts between algorithms and material, sensory, and physical dimensions, providing important insights for designing fair algorithms.

• Experimental or Evaluation Results:

  • Performers demonstrated the ability to respond to the algorithm's "misunderstanding" of human limitations, such as creatively interpreting and modifying original instructions to mitigate their harshness.
  • The study clearly revealed that "collaborating with algorithms" is a false proposition in the context of current large language models, as algorithms lack genuine intentions and feedback capabilities.

• Limitations and Future Directions:

  • Limitations:
    • Only five performance artists participated, with limited sample size and diversity.
    • The study focused on the GPT-3 model, which may not fully generalize to other algorithms.
  • Future Directions:
    • Expanding research to diverse populations and contexts, focusing on how different individuals "reinterpret" algorithmic behavior.
    • Exploring mechanisms that allow users to resist algorithmic rules, reducing social risks.

Through this study, the authors proposed a multidimensional analytical framework for human-algorithm interaction, while cautioning against the ethical and practical consequences of considering "human-like mimicry" in algorithm design. This provides important references for future technology design and policy-making.