Coaching Copilot: Blended Form of an LLM-Powered Chatbot and a Human Coach to Effectively Support Self-Reflection for Leadership GrowthChatbots' role in fostering self-reflection is now widely recognized, especially in inducing users' behavior change. While the benefits of 24/7 availability, scalability, and consistent responses have been demonstrated in contexts such as healthcare and tutoring to help one form a new habit, their utilization in coaching necessitating deeper introspective dialogue to induce leadership growth remains unexplored. This paper explores the potential of such a chatbot powered by recent Large Language Models (LLMs) in collaboration with professional coaches in the field of executive coaching. Through a design workshop with them and two weeks of user study involving ten coach-client pairs, we explored the feasibility and nuances of integrating chatbots to complement human coaches. Our findings highlight the benefits of chatbots' ubiquity and reasoning capabilities enabled by LLMs while identifying their limitations and design necessities for effective collaboration between human coaches and chatbots. By doing so, this work contributes to the foundation for augmenting one's self-reflective process with prevalent conversational agents through the human-in-the-loop approach.2024RARiku Arakawa et al.Conversational ChatbotsHuman-LLM CollaborationCUI
Exposed or Erased: Algorithmic Censorship of Nudity in ArtThe intersection between art and technology poses new challenges for creative expression in the digital space. This paper investigates the algorithmic censorship of artistic nudity in social platforms by means of a qualitative study via semi-structured interviews with 14 visual artists who have experienced censorship online. We explore the professional, emotional, financial and artistic consequences of content removal or shadow-banning. Focusing on the concept of artistic nudity, our findings emphasize the significant impact on artists of the algorithmic censorship of art, the need to consider art as a special case to safeguard the freedom of expression, the importance of education, the limitations of today's content moderation algorithms and the pressing need for transparency and recourse mechanisms. We advocate for a multi-stakeholder governance model conducive to a more supportive, safer and inclusive online environment that respects and nurtures human creativity.2024PRPiera Riccio et al.ELLIS AlicanteAI Ethics, Fairness & AccountabilityContent Moderation & Platform GovernanceOnline Identity & Self-PresentationCHI
PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and UncertaintyA user often needs training and guidance while performing several daily life procedures, e.g., cooking, setting up a new appliance, or doing a COVID test. Watch-based human activity recognition (HAR) can track users' actions during these procedures. However, out of the box, state-of-the-art HAR struggles from noisy data and less-expressive actions that are often part of daily life tasks. This paper proposes PrISM-Tracker, a procedure-tracking framework that augments existing HAR models with (1) graph-based procedure representation and (2) a user-interaction module to handle model uncertainty. Specifically, PrISM-Tracker extends a Viterbi algorithm to update state probabilities based on time-series HAR outputs by leveraging the graph representation that embeds time information as prior. Moreover, the model identifies moments or classes of uncertainty and asks the user for guidance to improve tracking accuracy. We tested PrISM-Tracker in two procedures: latte-making in an engineering lab study and wound care for skin cancer patients at a clinic. The results showed the effectiveness of the proposed algorithm utilizing transition graphs in tracking steps and the efficacy of using simulated human input to enhance performance. This work is the first step toward human-in-the-loop intelligent systems for guiding users while performing new and complicated procedural tasks. https://dl.acm.org/doi/10.1145/35695042023RARiku Arakawa et al.Human Pose & Activity RecognitionBiosensors & Physiological MonitoringUbiComp
CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative ModelsCatAlyst uses generative models to help workers’ progress by influencing their task engagement instead of directly contributing to their task outputs. It prompts distracted workers to resume their tasks by generating a continuation of their work and presenting it as an intervention that is more context-aware than conventional (predetermined) feedback. The prompt can function by drawing their interest and lowering the hurdle for resumption even when the generated continuation is insufficient to substitute their work, while recent human-AI collaboration research aiming at work substitution depends on a stable high accuracy. This frees CatAlyst from domain-specific model-tuning and makes it applicable to various tasks. Our studies involving writing and slide-editing tasks demonstrated CatAlyst’s effectiveness in helping workers swiftly resume tasks with a lowered cognitive load. The results suggest a new form of human-AI collaboration where large generative models publicly available but imperfect for each individual domain can contribute to workers’ digital well-being.2023RARiku Arakawa et al.Carnegie Mellon UniversityHuman-LLM CollaborationNotification & Interruption ManagementWorkplace Wellbeing & Work StressCHI
BeParrot: Efficient Interface for Transcribing Unclear Speech via RespeakingTranscribing speech from audio files to text is an important task not only for exploring the audio content in text form but also for utilizing the transcribed data as a source to train speech models, e.g., automated speech recognition (ASR) models. A post-correction approach has been frequently employed to reduce the time cost of transcription where users edit errors in the recognition results of ASR models. However, this approach assumes clear speech and is not designed for unclear speech (e.g., speech with high levels of noise or reverberation), which severely degrades the accuracy of ASR and requires many manual corrections. To construct an alternative approach to transcribe unclear speech, we introduce the idea of respeaking, which has primarily been used to create captions for television programs in real time. In respeaking, a proficient human respeaker repeats the heard speech as shadowing, and their utterances are recognized by an ASR model. While this approach can be effective for transcribing unclear speech, one problem is that respeaking is a highly cognitively demanding task and extensive training is often required to become a respeaker. We address this point with BeParrot, the first interface designed for respeaking that allows novice users to benefit from respeaking without extensive training through two key features, i.e, parameter adjustment and pronunciation feedback. Our user study involving 60 crowd workers demonstrated that they could transcribe different types of unclear speech 32.2 % faster with BeParrot than with a conventional approach without losing the accuracy of transcriptions. In addition, comments from the workers supported the design of the adjustment and feedback features, exhibiting a willingness to continue using BeParrot for transcription tasks. Our work demonstrates how we can leverage recent advances in machine learning techniques to overcome the area that is still challenging for computers themselves with the help of a human-in-the-loop approach.2022RARiku Arakawa et al.Intelligent Voice Assistants (Alexa, Siri, etc.)Conversational ChatbotsIUI
Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory PerturbationExplicitly alerting users is not always an optimal intervention, especially when they are not motivated to obey. For example, in video-based learning, learners who are distracted from the video would not follow an alert asking them to pay attention. Inspired by the concept of Mindless Computing, we propose a novel intervention approach, Mindless Attractor, that leverages the nature of human speech communication to help learners refocus their attention without relying on their motivation. Specifically, it perturbs the voice in the video to direct their attention without consuming their conscious awareness. Our experiments not only confirmed the validity of the proposed approach but also emphasized its advantages in combination with a machine learning-based sensing module. Namely, it would not frustrate users even though the intervention is activated by false-positive detection of their attentive state. Our intervention approach can be a reliable way to induce behavioral change in human-AI symbiosis.2021RARiku Arakawa et al.The University of Tokyo, ACES Inc.Privacy by Design & User ControlNotification & Interruption ManagementCHI
Personalised Recommendations in Mental Health Apps: The Impact of Autonomy and Data SharingThe recent growth of digital interventions for mental well-being prompts a call-to-arms to explore the delivery of personalised recommendations from a user's perspective. In a randomised placebo study with a two-way factorial design, we analysed the difference between an autonomous user experience as opposed to personalised guidance, with respect to both users’ preference and their actual usage of a mental well-being app. Furthermore, we explored users’ preference in sharing their data for receiving personalised recommendations, by juxtaposing questionnaires and mobile sensor data. Interestingly, self-reported results indicate the preference for personalised guidance, whereas behavioural data suggests that a blend of autonomous choice and recommended activities results in higher engagement. Additionally, although users reported a strong preference of filling out questionnaires instead of sharing their mobile data, the data source did not have any impact on the actual app use. We discuss the implications of our findings and provide takeaways for designers of mental well-being applications.2021SPSvenja Pieritz et al.AlphaRecommender System UXMental Health Apps & Online Support CommunitiesCHI
No More Handshaking: How have COVID-19 pushed the expansion of computer-mediated communication in Japanese idol culture?In Japanese idol culture, meet-and-greet events where fans were allowed to handshake with an idol member for several seconds were regarded as its essential component until the spread of COVID-19. Now, idol groups are struggling in the transition of such events to computer-mediated communication because these events had emphasized meeting face-to-face over communicating, as we can infer from their length of time. I anticipated that investigating this emerging transition would provide implications because their communication has a unique characteristic that is distinct from well-studied situations, such as workplace communication and intimate relationships. Therefore, I first conducted a quantitative survey to develop a precise understanding of the transition, and based on its results, had semi-structured interviews with idol fans about their perceptions of the transition. The survey revealed distinctive approaches, including one where fans gathered at a venue but were isolated from the idol member by an acrylic plate and talked via a video call. Then the interviews not only provided answers to why such an approach would be reasonable but also suggested the existence of a large gap between conventional offline events and emerging online events in their perceptions. Based on the results, I discussed how we can develop interaction techniques to support this transition and how we can apply it to other situations outside idol culture, such as computer-mediated performing arts.2021HYHiromu YakuraUniversity of TsukubaTeleoperation & TelepresenceInteractive Narrative & Immersive StorytellingCHI
INWARD: A Computer-Supported Tool for Video-Reflection Improves Efficiency and Effectiveness in Executive CoachingVideo-Reflection is a common approach to realize reflection in the field of executive coaching for professional development, which presents a video recording of the coaching session to a coachee in order to make the coachee reflectively think about oneself. However, it requires a great deal of time to watch the full length of the video and is highly dependent on the skills of the coach. We expect that the quality and efficiency of video-reflection can be improved with the support of computers. In this paper, we introduce INWARD, a computational tool that leverages human behavior analysis and video-based interaction techniques. The results of a user study involving 20 coaching sessions with five coaches indicate that INWARD enables efficient video-reflection and, by leveraging meta-reflection, realizes the ameliorated outcome of executive coaching. Moreover, discussions based on comments from the participants support the effectiveness of INWARD and suggest further possibilities of computer-supported approaches.2020RARiku Arakawa et al.The University of TokyoHuman Pose & Activity RecognitionInteractive Data VisualizationCHI
REsCUE: A framework for REal-time feedback on behavioral CUEs using multimodal anomaly detectionExecutive coaching has been drawing more and more attention for developing corporate managers. While conversing with managers, coach practitioners are also required to understand internal states of coachees through objective observations. In this paper, we present REsCUE, an automated system to aid coach practitioners in detecting unconscious behaviors of their clients. Using an unsupervised anomaly detection algorithm applied to multimodal behavior data such as the subject's posture and gaze, REsCUE notifies behavioral cues for coaches via intuitive and interpretive feedback in real-time. Our evaluation with actual coaching scenes confirms that REsCUE provides the informative cues to understand internal states of coachees. Since REsCUE is based on the unsupervised method and does not assume any prior knowledge, further applications beside executive coaching are conceivable using our framework.2019RARiku Arakawa et al.University of TokyoHuman Pose & Activity RecognitionComputational Methods in HCICHI
FocusMusicRecommender: A System for Recommending Music to Listen to While WorkingThis paper proposes FocusMusicRecommender, an automated system recommending background music to listen to while working. Recommendation systems matching user preferences have been widely researched even though research has shown that music that listeners strongly like is not suitable background music because it interferes with their concentration. FocusMusicRecommender plays songs that users may "neither like nor dislike" instead of "like very much." It is designed to by default summarize a song automatically so that users can give "like very much" feedback by pressing a "keep listening" button or "dislike very much" feedback by pressing a "skip" button. It uses this feedback, along with users' concentration levels estimated from their behavior history, to distinguish between the preference levels "like" and "like very much." It then estimates the preference levels of unplayed songs and selects the most suitable song by considering the user's current concentration level. The effectiveness of the proposed feedback method and suitability of the recommendation results were verified experimentally and in user studies.2018HYHiromu Yakura et al.Recommender System UXIUI
Maker Movements, Do-It-Yourself Cultures and Participatory Design: Implications for HCI Research.Falling costs and the wider availability of computational components, platforms and ecosystems have enabled the expansion of maker movements and DIY cultures. This can be considered as a form of democratization of technology systems design, in alignment with the aims of Participatory Design approaches. However, this landscape is constantly evolving, and long-term implications for the HCI community are far from clear. The organizers of this one-day workshop invite participants to present their case studies, experiences and perspectives on the topic with the goal of increasing understanding within this area of research. The outcomes of the workshop will include the articulation of future research directions with the purpose of informing a research agenda, as well as the establishment of new collaborations and networks.2018MSMichael Smyth et al.Edinburgh Napier UniversityMakerspace CultureParticipatory DesignComputational Methods in HCICHI