Stick-To-XR: Understanding Stick-Based User Interface Design for Extended RealityThis work explores the design of stick-shaped Tangible User Interfaces (TUI) for Extended Reality (XR). While sticks are widely used in everyday objects, their applications as a TUI in XR have not been systematically studied. We conducted a participatory design session with twelve experts in XR and Human-Computer Interaction to investigate the affordances of stick-based objects and how to utilize them in XR. As a result, we presented a taxonomy of stick-based objects' affordances in terms of their functions and holding gestures. Following that, we proposed four types of stick-based XR controller forms and discussed their advantages and limitations. In the end, we juxtaposed twenty-six existing XR controllers against our proposed forms and identified Landed (Cane) Stick, Thin Stick's flexible usages, and Modular Design as the major opportunities that remain unexamined yet for stick-based XR TUI design.2024YZYaying Zhang et al.Hand Gesture RecognitionMixed Reality WorkspacesDIS
AdaptiveVoice: Cognitively Adaptive Voice Interface for Driving AssistanceCurrent voice assistants present messages in a predefined format without considering users’ mental states. This paper presents an optimization-based approach to alleviate this issue which adjusts the level of details and speech speed of the voice messages according to the estimated cognitive load of the user. In the first user study (N=12), we investigated the impact of cognitive load on user performance. The findings reveal significant differences in preferred message formats across five cognitive load levels, substantiating the need for voice message adaptation. We then implemented AdaptiveVoice, an algorithm based on combinatorial optimization to generate adaptive voice messages in real-time. In the second user study (N=30) conducted in a VR-simulated driving environment, we compared AdaptiveVoice with a fixed format baseline, with and without visual guidance on the Heads-up display(HUD). Results indicate that users benefit from AdaptiveVoice with reduced response time and improved driving performance, particularly when it is augmented with HUD.2024SWShaoyue Wen et al.New York University, Xi'an Jiaotong-Liverpool UniversityHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Voice User Interface (VUI) DesignCHI
Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed RealityTangible interfaces in mixed reality (MR) environments allow for intuitive data interactions. Tangible cubes, with their rich interaction affordances, high maneuverability, and stable structure, are particularly well-suited for exploring multi-dimensional data types. However, the design potential of these cubes is underexplored. This study introduces a design space for tangible cubes in MR, focusing on interaction space, visualization space, sizes, and multiplicity. Using spatio-temporal data, we explored the interaction affordances of these cubes in a workshop (N=24). We identified unique interactions like rotating, tapping, and stacking, which are linked to augmented reality (AR) visualization commands. Integrating user-identified interactions, we created a design space for tangible-cube interactions and visualization. A prototype visualizing global health spending with small cubes was developed and evaluated, supporting both individual and combined cube manipulation. This research enhances our grasp of tangible interaction in MR, offering insights for future design and application in diverse data contexts.2024SHShuqi He et al.Xi'an Jiaotong - Liverpool UniversityMixed Reality WorkspacesInteractive Data VisualizationTime-Series & Network Graph VisualizationCHI
Exploration of Foot-based Text Entry Techniques for Virtual Reality EnvironmentsFoot-based input can serve as a supplementary or alternative approach to text entry in virtual reality (VR). This work explores the feasibility and design of foot-based techniques that are hands-free. We first conducted a preliminary study to assess foot-based text entry in standing and seated positions with tap and swipe input approaches. The findings showed that foot-based text input was feasible, with the possibility for performance and usability improvements. We then developed three foot-based techniques, including two tap-based techniques (FeetSymTap and FeetAsymTap) and one swipe-based technique (FeetGestureTap), and evaluated their performance via another user study. The results show that the two tap-based techniques supported entry rates of 11.12 WPM and 10.80 WPM, while the swipe-based technique led to 9.16 WPM. Our findings provide a solid foundation for the future design and implementation of foot-based text entry in VR and have the potential to be extended to MR and AR.2024TWTingjie Wan et al.Xi'an Jiaotong-Liverpool UniversityFoot & Wrist InteractionSocial & Collaborative VRImmersion & Presence ResearchCHI
Predicting Gaze-based Target Selection in Augmented Reality Headsets based on Eye and Head Endpoint DistributionsTarget selection is a fundamental task in interactive Augmented Reality (AR) systems. Predicting the intended target of selection in such systems can provide users with a smooth, low-friction interaction experience. Our work aims to predict gaze-based target selection in AR headsets with eye and head endpoint distributions, which describe the probability distribution of eye and head 3D orientation when a user triggers a selection input. We first conducted a user study to collect users’ eye and head behavior in a gaze-based pointing selection task with two confirmation mechanisms (air tap and blinking). Based on the study results, we then built two models: a unimodal model using only eye endpoints and a multimodal model using both eye and head endpoints. Results from a second user study showed that the pointing accuracy is improved by approximately 32% after integrating our models into gaze-based selection techniques.2023YWYushi Wei et al.Xi'an Jiaotong-Liverpool UniversityEye Tracking & Gaze InteractionAR Navigation & Context AwarenessCHI
Tasks of a Different Color: How Crowdsourcing Practices Differ per Complex Task Type and Why This MattersCrowdsourcing in China is a thriving industry. Among its most interesting structures, we find crowdfarms, in which crowdworkers self-organize as small organizations to tackle macrotasks. Little, however, is known as to which practices these crowdfarms use to tackle the macrotasks, and this goes hand in hand with the current practice of the HCI research community to treat all forms of complex crowdsourcing work as practically the same. However, macrotasks differ substantially regarding structure and decomposability. Treating them under one umbrella term - macrotasking - can lead to an imprecise understanding of the workforce involved. We address this gap by examining the work practices of 31 Chinese crowdfarms on the four main macrotask types, namely: modular, interlaced, wicked, and container macrotasks. Our results confirm essential differences in how these nascent crowd organizations address different macrotasks and shed light on what platforms can do to improve the uptake of such work.2023YWYihong Wang et al.Xi'an Jiaotong-Liverpool UniversityMental Health Apps & Online Support CommunitiesCrowdsourcing Task Design & Quality ControlDeveloping Countries & HCI for Development (HCI4D)CHI
iText: Hands-free Text Entry on an Imaginary Keyboard for Augmented Reality SystemsText entry is an important and frequent task in interactive devices including augmented reality head-mounted displays (AR HMDs). In current AR HMDs, there are still two main open challenges to overcome for efficient and usable text entry: arm fatigue due to mid-air input and visual occlusion because of their small see-through displays. To address these challenges, we present iText, a technique for AR HMDs that is hands-free and is based on an imaginary (invisible) keyboard. We first show that it is feasible and practical to use an imaginary keyboard on AR HMDs. Then, we evaluated its performance and usability with three hands-free selection mechanisms: eye blinks (E-Type), dwell (D-Type), and swipe gestures (G-Type). Our results show that users could achieve an average text entry speed of 11.95, 9.03 and 9.84 words per minutes (WPM) with E-Type, D-Type, and G-Type, respectively. Given that iText with E-Type outperformed the other two selection mechanisms in text entry rate and subjective feedback, we ran a third, 5-day study. Our results show that iText with E-Type can achieve an average text entry rate of 13.76 WPM with a mean word error rate of 1.5\%. In short, iText can enable efficient eyes-free text entry and can be useful for various application scenarios in AR HMDs.2021XLXueshi Lu et al.Voice User Interface (VUI) DesignAR Navigation & Context AwarenessUIST
Gaze-Supported 3D Object Manipulation in Virtual RealityThis paper investigates integration, coordination, and transition strategies of gaze and hand input for 3D object manipulation in VR. Specifically, this work aims to understand whether incorporating gaze input can benefit VR object manipulation tasks, and how it should be combined with hand input for improved usability and efficiency. We designed four gaze-supported techniques that leverage different combination strategies for object manipulation and evaluated them in two user studies. Overall, we show that gaze did not offer significant performance benefits for transforming objects in the primary working space, where all objects were located in front of the user and within the arm-reach distance, but can be useful for a larger environment with distant targets. We further offer insights regarding combination strategies of gaze and hand input, and derive implications that can help guide the design of future VR systems that incorporate gaze input for 3D object manipulation.2021DYDifeng Yu et al.The University of MelbourneEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
Distractor Effects on Crossing-Based InteractionTask-irrelevant distractors affect visuo-motor control for target acquisition and studying such effects has already received much attention in human-computer interaction. However, there has been little research into distractor effects on crossing-based interaction. We thus conducted an empirical study on pen-based interfaces to investigate six crossing tasks with distractor interference in comparison to two tasks without it. The six distractor-related tasks differed in movement precision constraint (directional/amplitude), target size, target distance, distractor location and target-distractor spacing. We also developed and experimentally validated six quantitative models for the six tasks. Our results show that crossing targets with distractors had longer average times and similar accuracy than that without distractors. The effects of distractors varied depending on distractor location, target-distractor spacing and movement precision constraint. When spacing is smaller than 11.27 mm, crossing tasks with distractor interference can be regarded as pointing tasks or a combination of pointing and crossing tasks, which could be better fitted with our proposed models than Fitts' law. According to these results, we provide practical implications to crossing-based user interface design.2021HTHuawei Tu et al.La Trobe UniversityUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI
Effect of Gameplay Uncertainty, Display Type, and Age on Virtual Reality ExergamesUncertainty is widely acknowledged as an engaging gameplay element but rarely used in exergames. In this research, we explore the role of uncertainty in exergames and introduce three uncertain elements (false-attacks, misses, and critical hits) to an exergame. We conducted a study under two conditions (uncertain and certain), with two display types (virtual reality and large display) and across young and middle-aged adults to measure their effect on game performance, experience, and exertion. Results show that (1) our designed uncertain elements are instrumental in increasing exertion levels; (2) when playing a motion-based first-person perspective exergame, virtual reality can improve performance, while maintaining the same motion sickness level as a large display; and (3) exergames for middle-aged adults should be designed with age-related declines in mind, similar to designing for elderly adults. We also framed two design guidelines for exergames that have similar features to the game used in this research.2021WXWenge Xu et al.Xi’an Jiaotong-Liverpool UniversityFull-Body Interaction & Embodied InputGame UX & Player BehaviorSerious & Functional GamesCHI
DMove: Directional Motion-based Interaction for Augmented Reality Head-Mounted DisplaysWe present DMove, directional motion-based interaction for Augmented Reality (AR) Head-Mounted Displays (HMDs) that is both hands- and device-free. It uses directional walk-ing as a way to interact with virtual objects. To use DMove, a user needs to perform directional motions such as mov-ing one foot forward or backward. In this research, we first investigate the recognition accuracy of the motion direc-tions of our method and the social acceptance of this type of interactions together with users' comfort rating for each direction. We then optimize its design and conduct a sec-ond study to compare DMove in task performance and user preferences (workload, motion sickness, user experience), with two approaches—Hand interaction (Meta 2-like) and Head+Hand interaction (HoloLens-like) for menu selection tasks. Based on the results of these two studies, we provide a set of guidelines for DMove and further demonstrate two applications that utilize directional motions.2019WXWenge Xu et al.Xi'an Jiaotong-Liverpool UniversityFull-Body Interaction & Embodied InputAR Navigation & Context AwarenessCHI