Memory Manipulations in Extended Reality

Title of the Paper

Memory Manipulations in Extended Reality

Paper Information

• Subject Area: Human memory limitations and XR (Extended Reality) interaction technologies
• Keywords: XR memory manipulation, perceptual manipulation, XR design, extended reality, virtual reality, augmented reality

Research Background and Issues

1. Issues and Challenges:

   • Human memory has limitations, such as forgetting, distortion, and persistent memory problems, which significantly impact cognitive functions and social interactions.
   • Existing technologies (e.g., calendars, AI-assisted tools) primarily focus on compensating for memory failures but rarely explore how these limitations could be leveraged to manipulate memory.
   • The widespread adoption of XR (Extended Reality) technologies raises concerns about potential memory manipulation, such as influencing memory through virtual and augmented reality environments.

2. Importance:

   • Memory shapes individual self-perception, social behavior, and decision-making capabilities; thus, understanding how XR can manipulate memory and addressing the ethical challenges of such technologies is crucial.
   • XR, as a powerful medium, may be more effective in manipulating user memory compared to traditional media (e.g., images or text).

3. Research Motivation:

   • To explore the characteristics and advantages of XR in memory manipulation.
   • To understand how XR can influence long-term memory by enhancing or distorting it.
   • To evaluate the ethical and societal risks associated with XR technologies in facilitating "memory manipulation."

Solutions

1. Research Methods:

   • Conducted three speculative design workshops (n=12) focusing on experts in the fields of XR and memory, resulting in the design of 48 XR memory manipulation scenarios.
   • Applied thematic analysis to extract five core components from these scenarios, defining the framework for XR memory manipulations (XRMMs) and categorizing them into three main types.

2. Key Technologies and Framework:

   • Core Components: Include manipulation context, memory mechanisms, implementation techniques, memory functions, and ultimate goals.
   • Categories:
     • Encoding Stage: Using augmented reality (AR) technologies to influence the quality of memory input, such as by inducing emotions or directing attention.
     • Retrieval Stage: Utilizing virtual reality (VR) technologies to reconstruct past scenes, thereby distorting memory or influencing users' perceptions of events.
     • Pre-retrieval Stage: Employing AR or VR to induce forgetting by hiding memory cues or triggering stress to create memory blocking.

3. Innovations:

   • Clearly categorized XR memory manipulations (XRMMs), distinguishing them from traditional perceptual manipulation technologies (e.g., physical-virtual perceptual manipulations, VPPMs).
   • Proposed that XR memory manipulations can directly influence users' long-term behavior through the three autobiographical functions of memory (self-function, social function, and directive function).

Research Results

1. Specific Outcomes:

   • Defined the concept of XR memory manipulations (XRMM): the use of perceptual manipulation technologies based on memory limitations to influence users' memory over the long term.
   • Presented the five core components and three main categories (encoding, retrieval, pre-retrieval) of XRMM.

2. Experimental and Evaluation Results:

   • Showcased 54 speculative XR manipulation scenarios, applied in areas such as education, therapy, memory assistance, and commercial services.
   • Explored how emotional induction or attention manipulation can enhance memory, while revealing potential negative applications, such as malicious memory distortion or psychological manipulation.

3. Advantages:

   • XR technologies are more effective in manipulating memory compared to traditional media.
   • XR creates stronger immersion, enhancing the effectiveness of memory during encoding or retrieval processes.

4. Limitations and Future Directions:

   • Limitations:
     • Speculative design methods may produce overly fictional scenarios.
     • The collected scenarios primarily originate from Western countries, lacking a perspective of cultural diversity.
     • Collaborative coding was not objectively measured but employed coordinated approaches.
   • Future Directions:
     • Further exploration of XRMM technologies in practical applications such as education, psychological therapy, and human memory enhancement.
     • Strengthening discussions on how XR technologies may disrupt "perceptual integrity" and addressing ethical challenges.
     • Promoting standardization in XR technology design to reduce potential malicious use.

Conclusion

The paper provides a comprehensive analysis of XR memory manipulations from both technological and ethical perspectives, proposing a complete framework and classification method to help academia and industry better understand and address the profound impacts of XR technologies on the memory domain. It also calls for interdisciplinary collaboration to tackle future challenges.