Papers

"Tech'' a Deep Breath: Technology-guided Breathing Practise With or Without a Social Robot for Psychological and Emotional Well-Being

Most mental health conditions emerge during adolescence, making university years pivotal for intervention. Nearly 30% of students worldwide experience mental health difficulties, yet support remains constrained by stigma and limited resources. This work investigates how interactive technologies, integrated with a wearable heart-rate sensor, can enhance psychological and emotional well-being through the ancient yogic breathing practice, Nadi Shuddhi. We developed two autonomous, adaptive systems - one combining a social robot with a tablet, and one tablet-only, both delivering real-time guidance based on heart-rate variability and breath rate. A study involving 42 university students across 200 sessions revealed both systems significantly increased parasympathetic activation, mindfulness, and calmness, while reducing short-term stress and depression symptoms. Compared to a tablet-only condition, the robot's physical presence led to a significant decrease in breath rate, improved mood, higher competence, and more positive user perceptions, while usability remained comparable, highlighting its potential for supporting youth mental health through social robots with biofeedback.

Initiating the Global AI Dialogues: Laypeople Perspectives on the Future Role of genAI in Society from Nigeria, Germany and Japan

With the rapid development and release of generative AI (genAI) applications, policy discourses primarily take place on an expert level. Little space is given to laypeople - who have to adapt to and adopt the genAI innovations - to share their opinions and experiences. Addressing this gap, we organized 6h/3.5h laypeople dialogues in Nigeria, Japan, and Germany in July and August 2024. During the dialogues, participants discussed what a desirable future in light of genAI development could look like in one of three contexts: education, public service, and arts & culture. Participants explored the consequences of technology deployment, assessed the risks, mapped stakeholders, and derived measures to achieve a desirable goal. This study contributes to policy debates on genAI by providing recommendations derived from participants' identified requirements and suggested measures for genAI to create value and to foster a socially desirable future. We reflect on the results through a cross-national lens.

Predicting Signal Reception Information from GNSS Satellites in Indoor Environments without Site Survey: Towards Opportunistic Indoor Positioning based on GNSS Fingerprinting

Investigating Effect of Altered Auditory Feedback on Self-Representation, Subjective Operator Experience, and Task Performance in Teleoperation of a Social Robot

Teleoperating social robots requires operators to ``speak as the robot,'' as local users would favor robots whose appearance and voice match. This study focuses on real-time altered auditory feedback (AAF), a method to transform the acoustic traits of one's speech and provide feedback to the speaker, to transform the operator's self-representation toward ``becoming the robot.'' To explore whether AAF with voice transformation (VT) matched to the robot's appearance can influence the operator's self-representation and ease the task, we experimented with three conditions: no VT (No-VT), only VT (VT-only), and VT with AAF (VT-AAF), where participants teleoperated a robot to verbally serve real passersby at a bakery. The questionnaire results demonstrate that VT-AAF changed the participants' self-representation to match the robot's character and improved participants' subjective teleoperating experience, while task performance and implicit measures of self-representation were not significantly affected. Notably, 87\% of the participants preferred VT-AAF the most.

Designing a Multisensory VR Game Prototype for Older Adults - the Acceptability and Design Implications

Simultaneous declines in visual function (e.g., dynamic visual acuity), cognitive ability (e.g., cognitive control/multitasking), and physical function (e.g., balance) are major symptoms of aging. Integrating stimulation for those sensory channels into a game could be a suitable way for older adults to engage in long-term health interventions. However, existing game design has not considered the relationship and synergistic impact of multisensory channels of dynamic visual acuity, cognitive ability, and physical function for older adults. We therefore developed the first multisensory VR game system prototype based on cognitive psychology paradigms (e.g., multitasking and Go/No-Go tasks), full-body movement (limb movement), and dynamic visual acuity exercises (horizontal, vertical and forward-backward eye movements) in the VR system environment. We then conducted an experiment to measure the acceptability (in terms of e.g., cybersickness, mental workload, etc.) of our VR game for older adults. The young adults and a PC task were included for comparisons. Qualitative and quantitative results showed that older adults did not experience cybersickness in either sitting or standing postures during the VR gameplay; they well-accepted the workload of the VR game compared to the PC task. Our findings revealed that the design combination of three sensory channels shows synergistic benefits for older adults. Our game encourages older adults to engage in extensive body movement in sitting and standing postures, this is particularly important to people with disabilities who cannot stand. Design implications are provided for the future development and implementation of VR game design for older adults. Our work provides empirical support for the acceptability of multisensory VR systems in older adults, and contributes to the future design of VR games for older adults.

GPS-assisted Indoor Pedestrian Dead Reckoning

Indoor pedestrian dead reckoning (PDR) using embedded inertial sensors in smartphones has been actively studied in the ubicomp community. However, PDR relying only on inertial sensors suffers from the accumulation of errors from the sensors. Researchers have employed various indoor landmarks detectable by smartphone sensors such as magnetic fingerprints caused by elevators and Bluetooth signals from beacons with known coordinates to compensate for the errors. This study proposes a new type of indoor landmark that does not require additional device installation, e.g., beacons, and training data collection in a target environment, e.g., magnetic fingerprints, unlike existing landmarks. This study proposes the use of GPS signals received by a smartphone to correct the accumulated errors of the PDR. While it is impossible to locate the smartphone indoors using GPS satellites, the smartphone can receive signals at a window-side area through windows from satellites aligned with the orientation of the window normal. Based on this idea, we design a machine-learning-based module for detecting the proximity of a user to a window and the orientation of the window, which enables us to roughly determine the absolute coordinates of the smartphone and to correct the accumulated errors by referring to positions of window-side areas found in the floor plan of the environment. A key technical contribution of this study is designing the module, such that it can be trained based on data from environments other than the target environment yet work in any environment by extracting GPS-related information independent of wall orientation. We evaluated the effectiveness of the proposed method using sensor data collected in real environments. https://dl.acm.org/doi/10.1145/3569467

Out for In!: Empirical Study on the Combination Power of Two Service Robots for Product Recommendation

Service robots have increasingly been investigated in retailing. Previous studies mainly focused on the effectiveness of recommendation with regard to a single robot, and whether and how the use of two robots combined can achieve better performance remain unclear. In this study, we address this by exploring the combination power of two service robots for product recommendation in a bakery. We placed one robot inside the store for product recommendation and the other robot outside to promote the inside robot. Particularly, we are interested in the effects of the outside robot on the inside robot's performance in product recommendation. Our results indicate that using the outside robot to promote the inside robot achieved more purchases over using the inside robot alone. Particularly, we discovered that the outside robot increased the attention of customers toward the inside robot; hence, more customers checked and purchased the products. Based on the findings, we discuss the important points for the effective use of service robots.

XR-LIVE: Enhancing Asynchronous Shared-Space Demonstrations with Spatial-temporal Assistive Toolsets for Effective Learning in Immersive Virtual Laboratories

An immersive virtual laboratory (VL) could offer flexibility of time and space, as well as safety, for remote students to conduct laboratory activities through online experiential learning. Recording an instructor’s demonstration inside a VL is an approach that allows students to learn directly from a demonstration. However, students have to learn from a recording while controlling the playback, which requires additional spatial and temporal attention. This additional attention load could lead to mistakes in following laboratory procedures. We have identified four design requirements to reduce attention load in VLs; namely, organized learning steps, improved student sense of co-presence, reduction of task-instructor split-attention, and learning independent of interpersonal distance. Based on these requirements, we have designed and implemented spatial-temporal assistive toolsets for laboratories in virtual environment, namely XR-LIVE, that reduces mental load and enhance effective learning in an asynchronous shared-space demonstration, implemented based on the setup of a standard civil engineering laboratory. We also analyzed students’ behavior in the VL demonstration to design guidelines applicable to generic VLs.

interiqr: Unobtrusive Edible Tags using Food 3D Printing

We present interiqr, a method that utilizes the infill parameter in the 3D printing process to embed information inside the food that is difficult to recognize with the human eye. Our key idea is to utilize the air space or secondary materials to generate a specific pattern inside the food without changing the model geometry. As a result, our method exploits the patterns that appear as hidden edible tags to store the data and simultaneously adds them to a 3D printing pipeline. Our contribution also includes the framework that connects the user with a data-embedding interface through the food 3D printing process, and the decoding system allows the user to decode the information inside the 3D printed food through backlight illumination and a simple image processing technique. Finally, we evaluate the usability of our method under different settings and demonstrate our method through the example application scenarios.

Knowledge Graph Completion-based Question Selection for Acquiring Domain Knowledge through Dialogues

Building a perfect knowledge base in a certain domain is practically impossible, so it is effective for dialogue systems to acquire knowledge for enhancing an imperfect knowledge base through natural language dialogues with users. This paper proposes a framework for selecting questions for such knowledge acquisition when a knowledge graph is used as the knowledge base. The framework exploits knowledge graph completion (KGC) for predicting new links that are likely to be correct and selects questions on the basis of the KGC scores. One of the problems with this framework is that questions with incorrect content might be selected, which often occur when the link prediction performance is low, and this would reduce the users’ willingness to engage in the dialogues. To alleviate this problem, this paper presents two modifications to the KGC training: 1) creating pseudo entities having substrings of the names of the entities in the graph so that the entities whose names share substrings are connected and 2) limiting the range of negative sampling. Cross validation-based experiments we conducted showed that these modifications improved KGC performance. We also conducted a user study with crowdsourcing to investigate the subjective perception of the correctness of the predicted links. The results suggest that the model trained with the modifications is capable of avoiding questions with incorrect content.

Programmable Filament: Printed Filaments for Multi-material 3D Printing

From full-color objects to functional capacitive artifacts, 3D printing multi-materials became essential to broaden the application areas of digital fabrication. We present Programmable Filament, a novel technique that enables multi-material printing using a commodity FDM 3D printer, requiring no hardware upgrades. Our technique builds upon an existing printing technique in which multiple filament segments are printed and spliced into a single threaded filament. We propose an end-toend pipeline for 3D printing an object in multi-materials, with an introduction of the design systems for end-users. Optimized for low-cost, single-nozzle FDM 3D printers, the system is built upon our computational analysis and experiments to enhance its validity over various printers and materials to design and produce a programmable filament. Finally, we discuss application examples and speculate the future with its potential, such as custom filament manufacturing on-demand.

FoodFab: Creating Food Perception Illusions using Food 3D Printing

Personalization of eating such that everyone consumes only what they need allows improving our management of food waste. In this paper, we explore the use of food 3D printing to create perceptual illusions for controlling the level of perceived satiety given a defined amount of calories. We present FoodFab, a system that allows users to control their food intake through modifying a food's internal structure via two 3D printing parameters: infill pattern and infill density. In two experiments with a total of 30 participants, we studied the effect of these parameters on users' chewing time that is known to affect people's feeling of satiety. Our results show that we can indeed modify the chewing time by varying infill pattern and density, and thus control perceived satiety. Based on the results, we propose two computational models and integrate them into a user interface that simplifies the creation of personalized food structures.

Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications

Appearance-based gaze estimation methods that only require an off-the-shelf camera have significantly improved but they are still not yet widely used in the human-computer interaction (HCI) community. This is partly because it remains unclear how they perform compared to model-based approaches as well as dominant, special-purpose eye tracking equipment. To address this limitation, we evaluate the performance of state-of-the-art appearance-based gaze estimation for interaction scenarios with and without personal calibration, indoors and outdoors, for different sensing distances, as well as for users with and without glasses. We discuss the obtained findings and their implications for the most important gaze-based applications, namely explicit eye input, attentive user interfaces, gaze-based user modelling, and passive eye monitoring. To democratise the use of appearance-based gaze estimation and interaction in HCI, we finally present OpenGaze (www.opengaze.org), the first software toolkit for appearance-based gaze estimation and interaction.

Efficient Information Sharing Techniques between Workers of Heterogeneous Tasks in 3D CVE

Collaboration between a helper and a worker in a 3D collaborative virtual environment usually requires real-time information sharing, since the worker relies on the timely assistance from the helper. In contrast, collaboration between workers requires them to shift their attention between independent tasks and dependent tasks. In worker-worker collaborations, a real-time updating technique could create excess information, which may be a distraction. In this paper, we compare different information sharing techniques and determine an efficient technique for the collaboration between workers. In our user experiment, participants performed a floor plan design task in a designer and engineer pairing on a desktop VR environment. The results showed that the proposed information sharing technique, in which objects are updated based on local users' actions, is more suitable than real-time updates. In addition, we discuss design implications that can be applied to different collaborative scenarios.

How Information Sharing about Care Recipients by Family Caregivers Impacts Family Communication

Previous research has shown that tracking technologies have the potential to help family caregivers optimize their coping strategies and improve their relationships with care recipients. In this paper, we explore how sharing the tracked data (i.e., caregiving journals and patient’s conditions) with other family caregivers affects home care and family communication. Although previous works suggested that family caregivers may benefit from reading the records of others, sharing patients’ private information might fuel negative feelings of surveillance and violation of trust for care recipients. To address this research question, we added a sharing feature to the previously developed tracking tool and deployed it for six weeks in the homes of 15 family caregivers who were caring for a depressed family member. Our findings show how the sharing feature attracted the attention of care recipients and helped the family caregivers discuss sensitive issues with care recipients.

Training Person-Specific Gaze Estimators from User Interactions with Multiple Devices

Learning-based gaze estimation has significant potential to enable attentive user interfaces and gaze-based interaction on the billions of camera-equipped handheld devices and ambient displays. While training accurate person- and device-independent gaze estimators remains challenging, person-specific training is feasible but requires tedious data collection for each target device. To address these limitations, we present the first method to train person-specific gaze estimators across multiple devices. At the core of our method is a single convolutional neural network with shared feature extraction layers and device-specific branches that we train from face images and corresponding on-screen gaze locations. Detailed evaluations on a new dataset of interactions with five common devices (mobile phone, tablet, laptop, desktop computer, smart TV) and three common applications (mobile game, text editing, media center) demonstrate the significant potential of cross-device training. We further explore training with gaze locations derived from natural interactions, such as mouse or touch input.

Visualizing Gaze Direction to Support Video Coding of Social Attention for Children with Autism Spectrum Disorder

This paper presents a novel interface to support video coding of social attention in assessment for children with Autism Spectrum Disorder. Video-based evaluation of social attention in therapeutic activity poses efforts for observers to find target behaviors while handling the ambiguity of attention. Despite the recent advances in computer vision-based gaze estimation methods, fully automatic recognition of social attention under diverse environments is still challenging. The goal of this work is to investigate an approach using automatic video analysis in a supportive manner for guiding human judgement. The proposed interface displays visualization of gaze estimation results on videos, and provides GUI supports to allow users to define social attention labels in the video timeline for facilitating agreements within observers. Throughout user studies and expert reviews, we show how the interface helps observers to perform video coding of social attention and how it can be improved for more efficient support.