CAIM: Development and Evaluation of a Cognitive AI Memory Framework for Long-Term Interaction with Intelligent AgentsLarge language models (LLMs) have advanced the field of artificial intelligence (AI) and are a powerful enabler for interactive systems. However, they still face challenges in long-term interactions that require adaptation towards the user as well as contextual knowledge and an understanding of the ever-changing environment. To overcome these challenges, holistic memory modeling is required to efficiently retrieve and store relevant information across sessions for accurate responses. Cognitive AI, which aims to simulate the human thought process in a computerized model, highlights interesting aspects, such as thoughts, memory mechanisms, and decision making, that can contribute towards improved memory modeling for LLMs. Inspired by these principles, we propose CAIM, a cognitive AI memory framework that models key aspects of human memory through a multi-agent architecture. Specialized LLM-based agents handle memory-related functions such as retrieval, contextual relevance evaluation, and memory maintenance. We compare CAIM against existing approaches, focusing on metrics such as retrieval accuracy, response correctness, and contextual coherence. The results demonstrate that CAIM outperforms baseline frameworks in different metrics, highlighting its context awareness and demonstrating its contribution to memory mechanisms for improving long-term human-AI interactions.2026RWRebecca Westhäußer et al.Mercedes-Benz AGHuman-LLM CollaborationExplainable AI (XAI)AI-Assisted Decision-Making & AutomationIUI
MIRAGE: Enabling Real-Time Automotive Mediated RealityTraffic is inherently dangerous, with around 1.19 million fatalities annually. Automotive Mediated Reality (AMR) can enhance driving safety by overlaying critical information (e.g., outlines, icons, text) on key objects to improve awareness, altering objects' appearance to simplify traffic situations, and diminishing their appearance to minimize distractions. However, real-world AMR evaluation remains limited due to technical challenges. To fill this sim-to-real gap, we present MIRAGE, an open-source tool that enables real-time AMR in real vehicles. MIRAGE implements 15 effects across the AMR spectrum of augmented, diminished, and modified reality using state-of-the-art computational models for object detection and segmentation, depth estimation, and inpainting. In an on-road expert user study (N=9) of MIRAGE, participants enjoyed the AMR experience while pointing out technical limitations and identifying use cases for AMR. We discuss these results in relation to prior work and outline implications for AMR ethics and interaction design.2026PJPascal Jansen et al.Ulm UniversityAutomated Driving Interface & Takeover DesignHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)In-Vehicle Haptic, Audio & Multimodal FeedbackCHI
Bumpy Ride? Understanding the Effects of External Forces on Spatial Interactions in Moving VehiclesAs the use of Head-Mounted Displays in moving vehicles increases, passengers can immerse themselves in visual experiences independent of their physical environment. However, interaction methods are susceptible to physical motion, leading to input errors and reduced task performance. This work investigates the impact of G-forces, vibrations, and unpredictable maneuvers on 3D interaction methods. We conducted a field study with 24 participants in both stationary and moving vehicles to examine the effects of vehicle motion on four interaction methods: (1) Gaze\&Pinch, (2) DirectTouch, (3) Handray, and (4) HeadGaze. Participants performed selections in a Fitts' Law task. Our findings reveal a significant effect of vehicle motion on interaction accuracy and duration across the tested combinations of Interaction Method $\times$ Road Type $\times$ Curve Type. We found a significant impact of movement on throughput, error rate, and perceived workload. Finally, we propose future research considerations and recommendations on interaction methods during vehicle movement.2025MSMarkus Sasalovici et al.Mercedes-Benz Tech Motion GmbH; Ulm University, Institute of Media InformaticsHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Motion Sickness & Passenger ExperienceCHI
Augmented Journeys: Interactive Points of Interest for In-Car Augmented RealityAs passengers spend more time in vehicles, the demand for non-driving related tasks (NDRTs) increases. In-car Augmented Reality (AR) has the potential to enhance passenger experiences by enabling interaction with the environment through NDRTs using world-fixed Points of Interest (POIs). However, the effectiveness of existing interaction techniques and visualization methods for in-car AR remains unclear. Based on a survey (N=110) and a pre-study (N=10), we developed an interactive in-car AR system using a video see-through head-mounted display to engage with POIs via eye-gaze and pinch. Users could explore passed and upcoming POIs using three visualization techniques: List, Timeline, and Minimap. We evaluated the system's feasibility in a field study (N=21). Our findings indicate general acceptance of the system, with the List visualization being the preferred method for exploring POIs. Additionally, the study highlights limitations of current AR hardware, particularly the impact of vehicle movement on 3D interaction.2025RSRobin Connor Schramm et al.Mercedes-Benz Tech Motion GmbH; RheinMain University of Applied SciencesHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Motion Sickness & Passenger ExperienceAR Navigation & Context AwarenessCHI
Blending the Worlds: An evaluation of World-Fixed Visual Appearances in Automotive Augmented RealityWith the transition to fully autonomous vehicles, non-driving related tasks (NDRTs) become increasingly important, allowing passengers to use their driving time more efficiently. In-car Augmented Reality (AR) gives the possibility to engage in NDRTs while also allowing passengers to engage with their surroundings, for example, by displaying world-fixed points of interest (POIs). This can lead to new discoveries, provide information about the environment, and improve locational awareness. To explore the optimal visualization of POIs using in-car AR, we conducted a field study (N = 38) examining six parameters: positioning, scaling, rotation, render distance, information density, and appearance. We also asked for intention of use, preferred seat positions and preferred automation level for the AR function in a post-study questionnaire. Our findings reveal user preferences and general acceptance of the AR functionality. Based on these results, we derived UX-guidelines for the visual appearance and behavior of location-based POIs in in-car AR.2025RSRobin Connor Schramm et al.Mercedes-Benz Tech Motion GmbH; RheinMain University of Applied SciencesHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)AR Navigation & Context AwarenessCHI
Assessing Augmented Reality Selection Techniques for Passengers in Moving Vehicles: A Real-World User StudyNowadays, cars offer many possibilities to explore the world around you by providing location-based information displayed on a 2D-Map. However, this information is often only available to front-seat passengers while being restricted to in-car displays. To propose a more natural way of interacting with the environment, we implemented an augmented reality head-mounted display to overlay points of interest onto the real world. We aim to compare multiple selection techniques for digital objects located outside a moving car by investigating head gaze with dwell time, head gaze with hardware button, eye gaze with hardware button, and hand pointing with gesture confirmation. Our study was conducted in a moving car under real-world conditions (N=22), with significant results indicating that hand pointing usage led to slower and less precise content selection while eye gaze was preferred by participants and performed on par with the other techniques.2023RSRobin Connor Schramm et al.Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)AR Navigation & Context AwarenessAutoUI
Unified Conversational Models with System-Initiated Transitions between Chit-Chat and Task-Oriented DialoguesSpoken dialogue systems (SDSs) have been separately developed under two different categories, task-oriented and chit-chat. The former focuses on achieving functional goals and the latter aims at creating engaging social conversations without special goals. Creating a unified conversational model that can engage in both chit-chat and task-oriented dialogue is a promising research topic in recent years. However, the potential ``initiative'' that occurs when there is a change between dialogue modes in one dialogue has rarely been explored. In this work, we investigate two kinds of dialogue scenarios, one starts from chit-chat implicitly involving task-related topics and finally switching to task-oriented requests; the other starts from task-oriented interaction and eventually changes to casual chat after all requested information is provided. We contribute two efficient prompt models which can proactively generate a transition sentence to trigger system-initiated transitions in a unified dialogue model. One is a discrete prompt model trained with two discrete tokens, the other one is a continuous prompt model using continuous prompt embeddings automatically generated by a classifier. We furthermore show that the continuous prompt model can also be used to guide the proactive transitions between particular domains in a multi-domain task-oriented setting.2023YLYe Liu et al.Conversational ChatbotsHuman-LLM CollaborationAI-Assisted Decision-Making & AutomationCUI
Design Fiction in a Corporate Setting – a Case StudyWhile speculative methods (e.g., design fiction) could help industry to reflect critically about future products and services, little is known about their potentials and limits when applied in a corporate setting. In this case, we worked with an automobile manufacturer to explore the near future of self-driving cars. For more than a year, we speculated using diverse methods, developed stories about a future with self-driving cars, and told them through 24 artifacts (e.g., city map, quick start guides). In this paper, we scrutinize the process, discuss what has worked and what not, which challenges we faced, and how corporate requirements shaped the emerging design fiction. We saw a trend toward affirmative rather than speculative design, driven by product-centricity and the need to justify process and outcome through concrete innovation. While speculation raised interesting questions about technological futures, it remained a challenge to transform them into concrete design implications.2023RRRonda Ringfort-Felner et al.Automated Driving Interface & Takeover DesignDesign FictionDIS
Strives: String-based Force Feedback for Automotive EngineeringThe large potential of force feedback devices for interacting in Virtual Reality (VR) has been illustrated in a plethora of research prototypes. Yet, these devices are still rarely used in practice and it remains an open challenge how to move this research into practice. To that end, we contribute a participatory design study on the use of haptic feedback devices in the automotive industry. Based on a 10-month observing process with 13 engineers, we developed STRIVE, a string-based haptic feedback device. In addition to the design of STRIVE, this process led to a set of requirements for introducing haptic devices into industrial settings, which center around a need for flexibility regarding forces, comfort, and mobility. We evaluated STRIVE with 16 engineers in five different day-to-day automotive VR use cases. The main results show an increased level of trust and perceived safety as well as further challenges towards moving haptics research into practice.2021AAAlexander Achberger et al.Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Teleoperated DrivingForce Feedback & Pseudo-Haptic WeightUIST
Calibrating Pedestrians' Trust in Automated Vehicles: Does an Intent Display in an External HMI Support Trust Calibration and Safe Crossing Behavior?Policymakers recommend that automated vehicles (AVs) display their automated driving status using an external human-machine interface (eHMI). However, previous studies suggest that a status eHMI is associated with overtrust, which might be overcome by an additional yielding intent message. We conducted a video-based laboratory study (N=67) to investigate pedestrians’ trust and crossing behavior in repeated encounters with AVs. In a 2x2 between-subjects design, we investigated (1) the occurrence of a malfunction (AV failing to yield) and (2) system transparency (status eHMI vs. status+intent eHMI). Results show that during initial encounters, trust gradually increases and crossing onset time decreases. After a malfunction, trust declines but recovers quickly. In the status eHMI group, trust was reduced more, and participants showed 7.3 times higher odds of colliding with the AV as compared to the status+intent group. We conclude that a status eHMI can cause pedestrians to overtrust AVs and advocate additional intent messages.2021SFStefanie Martina Faas et al.Mercedes-Benz AG, Ulm UniversityExternal HMI (eHMI) — Communication with Pedestrians & CyclistsCHI
ExplAIn Yourself! Transparency for Positive UX in Autonomous DrivingIn a fully autonomous driving situation, passengers hand over the steering control to a highly automated system. Autonomous driving behaviour may lead to confusion and negative user experience. When establishing such new technology, the user’s acceptance and understanding are crucial factors regarding success and failure. Using a driving simulator and a mobile application, we evaluated if system transparency during and after the interaction can increase the user experience and subjective feeling of safety and control. We contribute an initial guideline for autonomous driving experience design, bringing together the areas of user experience, explainable artificial intelligence and autonomous driving. The AVAM questionnaire, UEQ-S and interviews show that explanations during or after the ride help turn a negative user experience into a neutral one, which might be due to the increased feeling of control. However, we did not detect an effect for combining explanations during and after the ride.2021TSTobias Schneider et al.Stuttgart Media UniversityAutomated Driving Interface & Takeover DesignExplainable AI (XAI)AI-Assisted Decision-Making & AutomationCHI
A Longitudinal Video Study on Communicating Status and Intent for Self-Driving Vehicle Pedestrian InteractionWith self-driving vehicles (SDVs), pedestrians cannot rely on communication with the driver anymore. Industry experts and policymakers are proposing an external Human-Machine Interface (eHMI) communicating the automated status. We investigated whether additionally communicating SDVs' intent to give right of way further improves pedestrians' street crossing. To evaluate the stability of these eHMI effects, we conducted a three-session video study with N=34 pedestrians where we assessed subjective evaluations and crossing onset times. This is the first work capturing long-term effects of eHMIs. Our findings add credibility to prior studies by showing that eHMI effects last (acceptance, user experience) or even increase (crossing onset, perceived safety, trust, learnability, reliance) with time. We found that pedestrians benefit from an eHMI communicating SDVs' status, and that additionally communicating SDVs' intent adds further value. We conclude that SDVs should be equipped with an eHMI communicating both status and intent.2020SFStefanie M. Faas et al.Mercedes-Benz AG & Ulm UniversityAutomated Driving Interface & Takeover DesignExternal HMI (eHMI) — Communication with Pedestrians & CyclistsCHI