Sustaining Human Agency, Attending to Its Cost: An Investigation into Generative AI Design for Non-Native Speakers' Language Use

Research Background and Problem

• Problem or Challenge: The authors focus on how to maintain human agency in AI-generated language environments, particularly addressing the issue of agency for non-native speakers when communicating through machine translation (MT). The study highlights that traditional machine translation interfaces offer very limited agency, leaving non-native speakers with minimal influence over translation outputs.
• Significance: With the widespread application of AI technologies in language generation, maintaining human agency under technological assistance has become a critical issue. Especially in cross-linguistic communication scenarios, preserving the authenticity and agency of non-native speakers' language expression holds practical significance for social integration and information exchange, particularly for immigrant communities.
• Research Motivation and Related Work:
  • Related literature indicates that AI-generated content can reduce human active participation, limiting their ability to express identity and values.
  • Post-editing features in professional translation have been extensively studied, but existing research often focuses on bilingual experts, failing to adequately address the agency issues faced by resource-constrained non-native speakers.
  • Studies suggest that non-native speakers experience a "recognition and generation" gap in their target language resources. This paper seeks to explore design possibilities for preserving agency in information communication scenarios.

Proposed Solution

• Proposed Design Approach: Based on Bandura's "agency-resource connection," the authors designed three human-machine translation interfaces to support non-native speakers' agency in language communication:

  1. Tagging Interface: Users can indirectly influence the content by evaluating the quality of translation outputs (e.g., clicking positive or negative tags).
  2. Standard Post-Editing Interface: Users can directly edit machine translation outputs, leveraging both their passive and active vocabulary resources.
  3. Enhanced Post-Editing Interface: Provides sentence suggestions generated by large language models (LLMs) to assist users in modifying translations more easily.

• Innovative Contributions:

  • Interface design tailored to the characteristics of non-native speakers' language resource composition (i.e., differences in recognition and generation capabilities), adjusting the scope of resource engagement.
  • While supporting agency, the design also reveals potential communication costs associated with enhanced agency (e.g., reduced depth of information exchange).

• Implementation Steps and Key Technologies:

  • Experimental Setup: 45 pairs of participants, each consisting of one non-native speaker (immigrant) and one native speaker, engaged in an online text-based information-seeking task.
  • Data Collection: Communication records from the tasks were analyzed for breadth and depth of communication, as well as alignment in information transfer. Subjective evaluations of agency and workload were also collected from participants.
  • Technical Support: Open-source machine translation models were used, combined with sentence suggestions provided by generative language models (LLMs).

Research Findings

• Specific Results:

  • Non-native speakers demonstrated significantly higher agency under the two post-editing conditions compared to the tagging condition, proving that editing translation content or accessing LLM-recommended resources can enhance agency.
  • However, excessive agency enhancement came at the cost of reduced communication depth and alignment, particularly under the two post-editing conditions.

• Advantages Over Existing Solutions:

  • The authors innovatively focus on the resource limitations of non-native speakers, proposing targeted designs that expand current research on agency and human-AI collaboration.
  • The experiments reveal the strengths and weaknesses of different designs, providing quantitative evidence for support.

• Experimental or Evaluation Results:

  • Under the tagging condition, communication depth was higher, and final alignment was better, but the communication scope was narrower.
  • Under the post-editing conditions, information exchange was broader, but depth and alignment both decreased.

• Limitations and Future Directions:

  • Limitations: The sample's linguistic background and task scenarios were relatively narrow, focusing only on Chinese-English translation. The findings may have limited applicability to other language pairs and communication contexts.
  • Future Directions:
    • Extend to other non-native speaker linguistic backgrounds and scenarios.
    • Investigate the impact of other human-computer interaction methods (e.g., voice translation) on agency preservation.
    • Explore design methods to efficiently balance agency enhancement and communication efficiency.
    • Train customized machine translation models using tagging and post-editing data to reflect users' language preferences and styles.

This study clearly identifies the challenges and opportunities in preserving agency during human-AI interactions through technical design, providing valuable insights for future design endeavors.