Predicting the Limits: Tailoring Unnoticeable Hand Redirection Offsets in Virtual Reality to Individuals’ Perceptual BoundariesMany illusion and interaction techniques in Virtual Reality (VR) rely on Hand Redirection (HR), which has proved to be effective as long as the introduced offsets between the position of the real and virtual hand do not noticeably disturb the user experience. Yet calibrating HR offsets is a tedious and time-consuming process involving psychophysical experimentation, and the resulting thresholds are known to be affected by many variables---limiting HR's practical utility. As a result, there is a clear need for alternative methods that allow tailoring HR to the perceptual boundaries of individual users. We conducted an experiment with 18 participants combining movement, eye gaze and EEG data to detect HR offsets Below, At, and Above individuals' detection thresholds. Our results suggest that we can distinguish HR At and Above from no HR. Our exploration provides a promising new direction with potentially strong implications for the broad field of VR illusions.2024MFMartin Feick et al.Full-Body Interaction & Embodied InputEye Tracking & Gaze InteractionBrain-Computer Interface (BCI) & NeurofeedbackUIST
Beyond the Blink: Investigating Combined Saccadic & Blink-Suppressed Hand Redirection in Virtual RealityIn pursuit of hand redirection techniques that are ever more tailored to human perception, we propose the first algorithm for hand redirection in virtual reality that makes use of saccades, i.e., fast ballistic eye movements that are accompanied by the perceptual phenomenon of change blindness. Our technique combines the previously proposed approaches of gradual hand warping and blink-suppressed hand redirection with the novel approach of saccadic redirection in one unified yet simple algorithm. We compare three variants of the proposed Saccadic & Blink-Suppressed Hand Redirection (SBHR) technique with the conventional approach to redirection in a psychophysical study (N=25). Our results highlight the great potential of our proposed technique for comfortable redirection by showing that SBHR allows for significantly greater magnitudes of unnoticeable redirection while being perceived as significantly less intrusive and less noticeable than commonly employed techniques that only use gradual hand warping.2024AZAndré Zenner et al.Saarland University, Saarland Informatics Campus, German Research Center for Artificial Intelligence (DFKI), Saarland Informatics CampusHand Gesture RecognitionFull-Body Interaction & Embodied InputEye Tracking & Gaze InteractionCHI
Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual DriversThe rapid advancement of the automotive industry towards automated and semi-automated vehicles has rendered traditional methods of vehicle interaction, such as touch-based and voice command systems, inadequate for a widening range of non-driving related tasks, such as referencing objects outside of the vehicle. Consequently, research has shifted toward gestural input (e.g., hand, gaze, and head pose gestures) as a more suitable mode of interaction during driving. However, due to the dynamic nature of driving and individual variation, there are significant differences in drivers' gestural input performance. While, in theory, this inherent variability could be moderated by substantial data-driven machine learning models, prevalent methodologies lean towards constrained, single-instance trained models for object referencing. These models show a limited capacity to continuously adapt to the divergent behaviors of individual drivers and the variety of driving scenarios. To address this, we propose IcRegress, a novel regression-based incremental learning approach that adapts to changing behavior and the unique characteristics of drivers engaged in the dual task of driving and referencing objects. We suggest a more personalized and adaptable solution for multimodal gestural interfaces, employing continuous lifelong learning to enhance driver experience, safety, and convenience. Our approach was evaluated using an outside-the-vehicle object referencing use case, which highlighted the superiority of the adapted incremental learning models over a single trained model across various driver traits such as handedness, driving experience, and numerous driving conditions. Finally, to facilitate reproducibility, ease deployment, and promote further research, we offer our approach as an open-source framework.2024AGAmr Gomaa et al.Hand Gesture RecognitionFull-Body Interaction & Embodied InputHuman Pose & Activity RecognitionIUI
Turn-It-Up: Rendering Resistance for Knobs in Virtual Reality through Undetectable Pseudo-HapticsRendering haptic feedback for interactions with virtual objects is an essential part of effective virtual reality experiences. In this work, we explore providing haptic feedback for rotational manipulations, e.g., through knobs. We propose the use of a Pseudo-Haptic technique alongside a physical proxy knob to simulate various physical resistances. In a psychophysical experiment with 20 participants, we found that designers can introduce unnoticeable offsets between real and virtual rotations of the knob, and we report the corresponding detection thresholds. Based on these, we present the Pseudo-Haptic Resistance technique to convey physical resistance while applying only unnoticeable pseudo-haptic manipulation. Additionally, we provide a first model of how C/D gains correspond to physical resistance perceived during object rotation, and outline how our results can be translated to other rotational manipulations. Finally, we present two example use cases that demonstrate the versatility and power of our approach.2023MFMartin Feick et al.Force Feedback & Pseudo-Haptic WeightImmersion & Presence ResearchUIST
VoxelHap: A Toolkit for Constructing Proxies Providing Tactile and Kinesthetic Haptic Feedback in Virtual RealityExperiencing virtual environments is often limited to abstract interactions with objects. Physical proxies allow users to feel virtual objects, but are often inaccessible. We present the VoxelHap toolkit which enables users to construct highly functional proxy objects using Voxels and Plates. Voxels are blocks with special functionalities that form the core of each physical proxy. Plates increase a proxy’s haptic resolution, such as its shape, texture or weight. Beyond pro- viding physical capabilities to realize haptic sensations, VoxelHap utilizes VR illusion techniques to expand its haptic resolution. We evaluated the capabilities of the VoxelHap toolkit through the construction of a range of fully functional proxies across a variety of use cases and applications. In two experiments with 24 participants, we investigate a subset of the constructed proxies, studying how they compare to a traditional VR controller. First, we investigated VoxelHap’s combined haptic feedback and second, the trade-offs of using ShapePlates. Our findings show that VoxelHap’s proxies outperform traditional controllers and were favored by participants.2023MFMartin Feick et al.In-Vehicle Haptic, Audio & Multimodal FeedbackShape-Changing Interfaces & Soft Robotic MaterialsImmersion & Presence ResearchUIST
Prototyping Autonomous Vehicle Windshields with AR and Real-Time Object Detection Visualization: An On-Road Wizard-of-Oz StudyAutonomous vehicles (AVs; SAE levels 4 and 5) face substantial challenges regarding acceptance and UX. Novel human-machine interfaces (HMIs) providing transparent system information could account for those and facilitate adoption. However, since the availability of AVs for early concept studies is limited, context-based interface prototyping is required. This paper demonstrates the prototype and wizard-of-oz-based on-road evaluation of a futuristic windshield HMI concept that visualizes real-time object detections via augmented reality (AR). In a mixed-methods within-subjects study (N=30), participants assessed three early-stage concept variants to explore whether object detection visualization can counteract the aforementioned challenges. The findings confirm that transparent system feedback can increase understandability, perceived usefulness, and hedonic UX, but the amount and the timing of the provided information are crucial. The applied prototyping method proved suitable for investigating HMI concepts with real-time AR on urban roads. Based on a critical discussion, the paper concludes with design and prototyping recommendations.2023LFLukas A. Flohr et al.Automated Driving Interface & Takeover DesignHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)AR Navigation & Context AwarenessDIS
Induce a Blink of the Eye: Evaluating Techniques for Triggering Eye Blinks in Virtual RealityAs more and more virtual reality (VR) headsets support eye tracking, recent techniques started to use eye blinks to induce unnoticeable manipulations to the virtual environment, e.g., to redirect users' actions. However, to exploit their full potential, more control over users' blinking behavior in VR is required. To this end, we propose a set of reflex-based blink triggers that are suited specifically for VR. In accordance with blink-based techniques for redirection, we formulate (i) effectiveness, (ii) efficiency, (iii) reliability, and (iv) unobtrusiveness as central requirements for successful triggers. We implement the soft- and hardware-based methods and compare the four most promising approaches in a user study. Our results highlight the pros and cons of the tested triggers, and show those based on the menace, corneal, and dazzle reflexes to perform best. From these results, we derive recommendations that help choosing suitable blink triggers for VR applications.2023AZAndré Zenner et al.Saarland University, Saarland Informatics Campus, German Research Center for Artificial Intelligence (DFKI), Saarland Informatics CampusEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
Designing Visuo-Haptic Illusions with Proxies in Virtual Reality: Exploration of Grasp, Movement Trajectory and Object MassVisuo-haptic illusions are a method to expand proxy-based interactions in VR by introducing unnoticeable discrepancies between the virtual and real-world. Yet, how different design variables affect the illusions with proxies is still unclear. To unpack a subset of variables, we conducted two user studies with 48 participants to explore the impact of (1) different grasping types and movement trajectories, and (2) different grasping types and object masses on the discrepancy which may be introduced. Our Bayes analysis suggests that grasping types and object masses (≤ 500 g) did not noticeably affect the discrepancy, but for movement trajectory, results were inconclusive. Further, we identified a significant difference between un-/restricted movement trajectories. Our data shows considerable differences in participants’ proprioceptive accuracy, which seem to correlate with their prior VR experience. Finally, we illustrate the impact of our key findings on the visuo-haptic illusion design process by showcasing a new design workflow.2022MFMartin Feick et al.Saarland Informatics CampusVibrotactile Feedback & Skin StimulationImmersion & Presence ResearchCHI
Capturing Tactile Properties of Real Surfaces for Haptic ReproductionTactile feedback of an object's surface enables us to discern its material properties and affordances. This understanding is used in digital fabrication processes by creating objects with high-resolution surface variations to influence a user's tactile perception. As the design of such surface haptics commonly relies on knowledge from real-life experiences, it is unclear how to adapt this information for digital design methods. In this work, we investigate replicating the haptics of real materials. Using an existing process for capturing an object's microgeometry, we digitize and reproduce the stable surface information of a set of 15 fabric samples. In a psychophysical experiment, we evaluate the tactile qualities of our set of original samples and their replicas. From our results, we see that direct reproduction of surface variations is able to influence different psychophysical dimensions of the tactile perception of surface textures. While the fabrication process did not preserve all properties, our approach underlines that replication of surface microgeometries benefits fabrication methods in terms of haptic perception by covering a large range of tactile variations. Moreover, by changing the surface structure of a single fabricated material, its material perception can be influenced. We conclude by proposing strategies for capturing and reproducing digitized textures to better resemble the perceived haptics of the originals.2021DDDonald Degraen et al.Electrical Muscle Stimulation (EMS)Shape-Changing Interfaces & Soft Robotic MaterialsUIST
Weirding Haptics: In-Situ Prototyping of Vibrotactile Feedback in Virtual Reality through VocalizationEffective haptic feedback in virtual reality (VR) is an essential element for creating convincing immersive experiences. To design such feedback, state-of-the-art VR setups provide APIs for programmatically generating controller vibration patterns. While tools for designing vibrotactile feedback keep evolving, they often require expert knowledge and rarely support direct manipulation methods for mapping feedback to user interactions within the VR environment. To address these challenges, we contribute a novel concept called Weirding Haptics, that supports fast-prototyping by leveraging the user's voice to design such feedback while manipulating virtual objects in-situ. Through a pilot study (N = 9) focusing on how tactile experiences are vocalized during object manipulation, we identify spatio-temporal mappings and supporting features needed to produce intended vocalizations. To study our concept, we built a VR design tool informed by the results of the pilot study. This tool enables users to design tactile experiences using their voice while manipulating objects, provides a set of modifiers for fine-tuning the created experiences in VR, and allows to rapidly compare various experiences by feeling them. Results from a validation study (N = 8) show that novice hapticians can vocalize experiences and refine their designs with the fine-tuning modifiers to match their intentions. We conclude our work by discussing uncovered design implications for direct manipulation and vocalization of vibrotactile feedback in immersive virtual environments.2021DDDonald Degraen et al.Haptic WearablesImmersion & Presence ResearchUIST
The Effect of Surrounding Scenery Complexity on the Transfer of Control Time in Highly Automated DrivingOne challenge in highly automated driving is the safe transfer of the control (ToC). A safe ToC requires estimating the take-over time depending on the driver's state in different environmental conditions, to adapt timing and design of the ToC request. We introduce the environmental complexity as one factor that affects the ToC time. In a driving simulator experiment (N=12), the participants drove in five scenes having different environmental complexities (i.e. density and height of the background objects) with and without a secondary task. The results revealed that the ToC time is proportional to the environmental complexity. Thus, in the same driving task and the same traffic, an increasing environmental complexity yields higher ToC times in both conditions with and without secondary task. Our model of environmental complexity is a first step towards a better prediction of ToC times in highly automated driving to measure the complexity of real environments.2021FWFrederik Wiehr et al.Automated Driving Interface & Takeover DesignIUI
Spinning Icons: Introducing a Novel SSVEP-BCI Paradigm Based on RotationSteady-State-Visually-Evoked-Potential (SSVEP) Brain-Computer Interfaces (BCIs) make use of flickering stimuli to determine the target a user is looking at and select commands accordingly. Those types of BCI can be operated with little to no training, achieve high classification accuracies and are robust in application. A drawback of this approach is the reduced user comfort due to the constant flickering of the stimuli which can be annoying and tiring to look at. Existing studies addressing this issue try to make use of motion to disguise the oscillating patterns. However, this makes them look abstract and restricts the design of those applications as those patterns do not blend in to conventional user interfaces. In this work we introduce the concept of spinning icons to evoke SSVEPs. The icons are rotating in a certain frequency around their vertical axis and are supposed to appear more natural and be less stressing for the human eye. Furthermore this concept is not bound to any kind of abstract motion based pattern but rather supposed to work with any type of icon or image. The newly designed stimuli were evaluated in an application-oriented scenario and compared to standard and state-of-the-art movement-based SSVEP stimuli regarding the classification accuracy and experienced visual fatigue. The results show that the newly created stimuli performed equally well and partially even better in terms of classification accuracy and were rated throughout better concerning visual fatigue by the study participants. This work therefore lays the foundation for more comfortable SSVEP-BCIs which can be used with basically every icon or UI element spinning around their vertical axis.2021MRMaurice Rekrut et al.Brain-Computer Interface (BCI) & NeurofeedbackIUI
Visuo-haptic Illusions for Linear Translation and Stretching using Physical Proxies in Virtual RealityProviding haptic feedback when manipulating virtual objects is an essential part of immersive virtual reality experiences; however, it is challenging to replicate all of an object’s properties and characteristics. We propose the use of visuo-haptic illusions alongside physical proxies to enhance the scope of proxy-based interactions with virtual objects. In this work, we focus on two manipulation techniques, linear translation and stretching across different distances, and investigate how much discrepancy between the physical proxy and the virtual object may be introduced without participants noticing. In a study with 24 participants, we found that manipulation technique and travel distance significantly affect the detection thresholds, and that visuo-haptic illusions impact performance and accuracy. We show that this technique can be used to enable functional proxy objects that act as stand-ins for multiple virtual objects, illustrating the technique through a showcase VR-DJ application.2021MFMartin Feick et al.Saarland Informatics CampusForce Feedback & Pseudo-Haptic WeightImmersion & Presence ResearchCHI
Enhancing Texture Perception in Virtual Reality Using 3D-Printed Hair StructuresExperiencing materials in virtual reality (VR) is enhanced by combining visual and haptic feedback. While VR easily allows changes to visual appearances, modifying haptic impressions remains challenging. Existing passive haptic techniques require access to a large set of tangible proxies. To reduce the number of physical representations, we look towards fabrication to create more versatile counterparts. In a user study, 3D-printed hairs with length varying in steps of 2.5 mm were used to influence the feeling of roughness and hardness. By overlaying fabricated hair with visual textures, the resolution of the user's haptic perception increased. As changing haptic sensations are able to elicit perceptual switches, our approach can extend a limited set of textures to a much broader set of material impressions. Our results give insights into the effectiveness of 3D-printed hair for enhancing texture perception in VR.2019DDDonald Degraen et al.Intel Visual Computing Institute (IVCI), Saarland Informatics CampusShape-Changing Interfaces & Soft Robotic MaterialsImmersion & Presence ResearchCHI
Drag:on - A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight ShiftStandard controllers for virtual reality (VR) lack sophisticated means to convey a realistic, kinesthetic impression of size, resistance or inertia. We present the concept and implementation of Drag:on, an ungrounded shape-changing VR controller that provides dynamic passive haptic feedback based on drag, i.e. air resistance, and weight shift. Drag:on leverages the airflow occurring at the controller during interaction. By dynamically adjusting its surface area, the controller changes the drag and rotational inertia felt by the user. In a user study, we found that Drag:on can provide distinguishable levels of haptic feedback. Our prototype increases the haptic realism in VR compared to standard controllers and when rotated or swung improves the perception of virtual resistance. By this, Drag:on provides haptic feedback suitable for rendering different virtual mechanical resistances, virtual gas streams, and virtual objects differing in scale, material and fill state.2019AZAndré Zenner et al.German Research Center for Artificial Intelligence (DFKI), Saarland Informatics CampusForce Feedback & Pseudo-Haptic WeightShape-Changing Interfaces & Soft Robotic MaterialsCHI
Gamified Ads: Bridging the Gap Between User Enjoyment and the Effectiveness of Online AdsWhile the use of ad blockers prevents negative impacts of advertising on user experience, it poses a serious threat to the business model of commercial web services and freely available content on the web. As an alternative, we investigate the user enjoyment and the advertising effectiveness of playfully deactivating online ads. We created eight game concepts, performed a pre-study assessing the users' perception of them (N=50) and implemented three well-perceived ones. In a lab study (N=72), we found that these game concepts are more enjoyable than deactivating ads without game elements. Additionally, one game concept was even preferred over using an ad blocker. Notably, playfully deactivating ads was shown to have a positive impact on users' brand and product memory, enhancing the advertising effectiveness. Thus, our results indicate that playfully deactivating ads is a promising way of bridging the gap between user enjoyment and effective advertising.2019MAMaximilian Altmeyer et al.German Research Center for Artificial Intelligence (DFKI), Saarland Informatics CampusRecommender System UXGame UX & Player BehaviorGamification DesignCHI
Multi-Modal Approaches for Post-Editing Machine TranslationCurrent advances in machine translation increase the need for translators to switch from traditional translation to post-editing (PE) of machine-translated text, a process that saves time and improves quality. This affects the design of translation interfaces, as the task changes from mainly generating text to correcting errors within otherwise helpful translation proposals. Our results of an elicitation study with professional translators indicate that a combination of pen, touch, and speech could well support common PE tasks, and received high subjective ratings by our participants. Therefore, we argue that future translation environment research should focus more strongly on these modalities in addition to mouse- and keyboard-based approaches. On the other hand, eye tracking and gesture modalities seem less important. An additional interview regarding interface design revealed that most translators would also see value in automatically receiving additional resources when a high cognitive load is detected during PE.2019NHNico Herbig et al.German Research Center for Artificial Intelligence (DFKI), Saarland Informatics CampusVoice User Interface (VUI) DesignHuman-LLM CollaborationCHI
"Enable or Disable Gamification?" - Analyzing the Impact of Choice in a Gamified Image Tagging TaskThis paper investigates a simple form of customization: giving users the choice to enable or disable gamification. We present a study (N=77) in the context of image tagging, in which a gamification approach was shown to be effective in previous work. In our case, some participants could enable or disable gamification after they had experienced the task with and without it. Other participants had no choice and did the task with or without game elements. The results indicate that those who are not attracted by the elements can be motivated to tag more through this choice. In contrast, those that like the elements are not affected by it. This suggests that systems should provide the option to disable gamification in the absence of more sophisticated tailoring.2019PLPascal Lessel et al.German Research Center for Artificial Intelligence (DFKI), Saarland Informatics CampusGamification DesignCHI