Lessons From Working in the Metaverse: Challenges, Choices, and Implications from a Case StudyAlthough the metaverse workspace has the potential to solve some of the drawbacks of remote work while maintaining its benefits, there are few real-world cases of adopting the metaverse as a legitimate workspace and fewer subsequent studies on how to design and operate the metaverse workspace. Thus, questions exist about the organizational or sociotechnical challenges that may emerge and how decisions are made when adopting and operating the metaverse workspace in a real-world setting. To answer such questions, we scrutinized the startup company Zigbang, which has completely replaced their physical office with Soma— a metaverse platform they developed where thousands of people work and other cooperative companies have moved in as tenants. By conducting field observations and semi-structured interviews with various workers and Zigbang’s stakeholders, we identify essential design challenges and decisions when adopting a metaverse workspace and highlight the key takeaways learned from the company’s trials and errors.2024HPHyanghee Park et al.Seoul National UniversityMixed Reality WorkspacesRemote Work Tools & ExperienceCHI
I feel being there, they feel being together: Exploring How Telepresence Robots Facilitate Long-Distance Family CommunicationMany families often live geographically apart from each other due to work, education, or marriage. Therefore, long-distance families frequently use computer-mediated communication (CMC) tools to stay connected. While CMC tools have significantly improved family communication, they cannot fully mediate social presence. To examine the potential of telepresence robots for improving long-distance family communication, we conducted a two-week qualitative in situ study involving eight families. We analyzed recorded videos of their family interactions and conducted pre- and post-deployment interviews. Our findings highlight telepresence robots' potential as family communication tools, enabling immersive, natural, and dynamic interactions through physical embodiment and autonomy. Particularly, we identified five categories of family interaction mediated by telepresence robots: engaging in multi-party family communication, exploring home, restoring family routines, providing support, and having joint physical activities. Based on our findings, we present design guidelines for leveraging telepresence robots as effective family communication tools.2024JSJiyeon Amy Seo et al.Seoul National UniversityTeleoperation & TelepresenceCHI
“Some Hope, Many Despair”: Experiences of the Normalization within Online Dating among Queer Women in a Closeted SocietyOnline dating technology mediates various social interactions for LGBTQ+ communities, yet how such technology shapes queerness remains understudied, particularly within queer women's communities in non-Western settings. To address this gap, we conducted a qualitative study with 17 queer women, aiming to uncover their experiences and challenges in online dating within the conservative context of South Korea. Contrary to their initial expectations of exploring open-ended forms of interaction, we found that dating applications tended to systematically normalize queerness in sexuality presentation, relationship building, and shared identities in the community. These mechanisms forced them to conform to the "normalized queerness," thereby impeding non-normative and flexible aspects of queer interactions. Building upon these findings, we discuss how the technological affordances of online dating platforms facilitate the normalization of queerness under the influence of sociocultural contexts of South Korea.2024SPSeora Park et al.Seoul National UniversityGender & Race Issues in HCILGBTQ+ Community Technology DesignCHI
Enhancing Auto-Generated Baseball Highlights via Win Probability and Bias Injection MethodThe automatic generation of sports highlight videos is emerging in both the sports entertainment domain and research community. Earlier methods for generating highlights rely on visual-audio cues or contextual cues, so they may not capture the overall flow of the game well. In this paper, we propose a technique based on Win Probability Added (WPA), an empirical sabermetric baseball statistic, to generate baseball highlights that can better reflect in-game dynamics. Additionally, we introduce methods for generating “biased” highlights toward one team by systematically manipulating WPAs. Through a mixed-method user study with 43 baseball enthusiasts, we found that participants evaluated WPA-based highlights more favorably than existing AI highlights. For (un)favorably biased highlights, the game result(win/loss) was the most dominating factor in user perception, but bias directions and strengths also had nuanced effects on them. Our work contributes to the development of automated tools for generating customized sports highlights.2024KPKieun Park et al.Seoul National UniversityRecommender System UXGame UX & Player BehaviorSerious & Functional GamesCHI
Investigating the Effects of Real-time Student Monitoring Interface on Instructors’ Monitoring Practices in Online TeachingThe shift to online education, accelerated by the COVID-19 pandemic, has introduced challenges in monitoring student engagement, an essential aspect of effective teaching. In response, real-time student monitoring interfaces have emerged as potential tools to aid instructors, yet their efficacy has not been thoroughly examined. Addressing this gap, we conducted a controlled experiment with 20 instructors examining the impact of engagement cues (presence versus absence) and student engagement levels (high versus low) on instructors' monitoring effectiveness, teaching behavior adjustments, and cognitive load in online classes. Our findings underscored the fundamental benefits of student engagement monitoring interfaces for improving monitoring quality and effectiveness. Furthermore, our study highlighted the critical need for customizable interfaces that could balance the informational utility of engagement cues with the associated cognitive load and psychological stress on instructors. These insights may offer design implications for the design of future student engagement monitoring interfaces.2024HLHa Yeon Lee et al.Seoul National UniversityOnline Learning & MOOC PlatformsCollaborative Learning & Peer TeachingCHI
The Power of Close Others: How Social Interactions Impact Older Adults' Mobile Shopping ExperienceIncreasingly, older adults are shopping via mobile devices as technology has been incorporated into their lives. When older adults adopt and use mobile shopping, social interactions with close others greatly influence their experience. Therefore, this paper aimed to provide a comprehensive understanding of how social interactions with close others shaped older adults' mobile shopping practices. We conducted in-depth semi-structured interviews with 31 older adults who reported using mobile shopping regularly. We found that older adults engaged in three types of social interaction: learning from, collaborating with, and assisting close others in adopting and using mobile shopping. Through these social interactions, they gradually built trust in mobile shopping systems and supported each other's decision-making processes. In conclusion, we presented design implications for facilitating social interactions to improve older adults' mobile shopping experience.2023JSJiyeon Amy Seo et al.Explainable AI (XAI)Aging-Friendly Technology DesignUniversal & Inclusive DesignDIS
DiVRsity: Design and Development of a Group Role-Play VR Platform for Disability Awareness EducationRole-playing can be an effective method for disability awareness education (DAE), and the immersive nature of virtual reality (VR) holds promise for enhancing DAE experiences. However, existing VR applications for DAE often pay less attention to the social aspects of disabilities, resulting in a lack of ability to simulate implicit social discrimination experienced by individuals with disabilities. To bridge this gap, we developed DiVRsity, a customizable VR group role-playing platform for DAE. To identify design requirements for DiVRsity, we conducted a formative study with VR and DAE experts. In an evaluation study, 28 participants engaged in role-playing exercises using DiVRsity, simulating discriminatory interpersonal situations experienced by individuals with vision impairments. Findings revealed that participants' disability awareness significantly increased after engaging in role-playing activities using DiVRsity compared to before. We discuss the potential of VR as a role-playing platform for DAE and provide design implications for future VR-based DAE tools.2023YJYewon Jin et al.Identity & Avatars in XRSpecial Education TechnologyInclusive DesignDIS
IntroBot: Exploring the Use of Chatbot-assisted Familiarization in Online Collaborative GroupsMany people gather online and form teams with strangers to collaborate on tasks. However, while intrateam trust and cohesion are critical for team performance, such characteristics take time to establish and are harder to build up through computer-mediated communication. Building on prior research that has shown that enhancing familiarity between members can help, we hypothesized that the use of a chatbot to support the familiarization of ad hoc teammates can help their collaboration. As such, we designed IntroBot, a chatbot that builds on an online discussion facilitator framework and leverages the social media data of users to assist their familiarization process. Through a between-subjects study (N=60), we found that participants who used IntroBot reported higher levels of trust, cohesion, and interaction quality, as well as generated more ideas in a collaborative brainstorming task. We discuss insights gained from our study, and present opportunities for the future of chatbot-assisted collaboration.2023DSDonghoon Shin et al.University of WashingtonConversational ChatbotsCollaborative Learning & Peer TeachingRemote Work Tools & ExperienceCHI
Towards a Metaverse Workspace: Opportunities, Challenges, and Design ImplicationsBoth enterprises and their employees have globally experienced remote work at an unprecedented scale since the outbreak of COVID-19. As the pandemic becomes less of a threat, some companies have called their employees back to a physical office, citing issues related to working remotely, but many employees have refused to return. Thus, working in the metaverse has gained much attention as an alternative that could complement the weaknesses of completely remote work or even offline work. However, we do not know yet what benefits and drawbacks the metaverse has as a legitimate workspace, because there are few real cases of 1) working in the metaverse and 2) working remotely at such an unprecedented scale. Thus, this paper aims to identify real challenges and opportunities the metaverse workspace presents when compared to remote work by conducting semi-structured interviews and participatory workshops with various employees and company stakeholders (e.g., HR managers and CEOs) who have experienced at least two of three work types: working in a physical office, remotely, or in the metaverse. Consequently, we identified 1) advantages and disadvantages of remote work and 2) opportunities and challenges of the metaverse. We further discuss design implications that may overcome the identified challenges of working in the metaverse.2023HPHyanghee Park et al.Seoul National UniversityMixed Reality WorkspacesCHI
Personalization Trade-offs in Designing Dialogue-based Interactive Information System for Support-Seeking of Sexual Violence SurvivorsThe lack of reliable, personalized information often complicates sexual violence survivors' support-seeking. Recently, there is an emerging approach to conversational information systems for support-seeking of sexual violence survivors, featuring personalization with wide availability and anonymity. However, a single best solution might not exist as sexual violence survivors have different needs and purposes in seeking support channels. To better envision conversational support-seeking systems for sexual violence survivors, we explore personalization trade-offs in designing such information systems. We implement a high-fidelity prototype dialogue-based information system through four design workshop sessions with three professional caregivers and interviewed with four self-identified survivors using our prototype. We then identify two forms of personalization trade-offs for conversational support-seeking systems: (1) specificity and sensitivity in understanding users and (2) relevancy and inclusiveness in providing information. To handle these trade-offs, we propose a reversed approach that starts from designing information and inclusive tailoring that considers unspecified needs, respectively.2022HKHyeok Kim et al.Northwestern UniversityConversational ChatbotsVoice AccessibilityEmpowerment of Marginalized GroupsCHI
Designing Fair AI in Human Resource Management: Understanding Tensions Surrounding Algorithmic Evaluation and Envisioning Stakeholder-Centered SolutionsEnterprises have recently adopted AI to human resource management (HRM) to evaluate employees’ work performance evaluation. However, in such an HRM context where multiple stakeholders are complexly intertwined with different incentives, it is problematic to design AI reflecting one stakeholder group’s needs (e.g., enterprises, HR managers). Our research aims to investigate what tensions surrounding AI in HRM exist among stakeholders and explore design solutions to balance the tensions. By conducting stakeholder-centered participatory workshops with diverse stakeholders (including employees, employers/HR teams, and AI/business experts), we identified five major tensions: 1) divergent perspectives on fairness, 2) the accuracy of AI, 3) the transparency of the algorithm and its decision process, 4) the interpretability of algorithmic decisions, and 5) the trade off between productivity and inhumanity. We present stakeholder-centered design ideas for solutions to mitigate these tensions and further discuss how to promote harmony among various stakeholders at the workplace.2022HPSoHyun Park et al.Seoul National UniversityAI-Assisted Decision-Making & AutomationAI Ethics, Fairness & AccountabilityAlgorithmic Transparency & AuditabilityCHI
Designing and Evaluating a Chatbot for Survivors of Image-Based Sexual AbuseImage-based sexual abuse (IBSA) is a severe social problem that causes survivors tremendous pain. IBSA survivors may encounter a lack of information and victim blame when seeking online and offline assistance. While institutions support survivors, they cannot be available 24 hours a day. Because the immediate reaction to IBSA is crucial to remove intimate images and prevent further distribution, survivors need first responders who are always accessible and do not blame them. Chatbots are constantly available, do not judge the conversation partner, and may deliver structured information and words of comfort. Therefore, we developed a chatbot to provide information and emotional support to IBSA survivors in dealing with their abuse. We analyzed nine chatbots for sexual violence survivors to identify common design elements. In addition, we sought advice from five professional counselors about the challenges survivors have while responding to their harm. We conducted a user study with 25 participants to determine the chatbot's effectiveness in providing information and emotional support compared to internet search. The chatbot was better than the internet search regarding information organization, accessibility, and conciseness. Furthermore, the chatbot excels in providing emotional support to survivors. We discuss the survivor-centered information structure and design consideration of emotionally supportive conversation.2022WMWookjae Maeng et al.Seoul National UniversityConversational ChatbotsMental Health Apps & Online Support CommunitiesOnline Harassment & Counter-ToolsCHI
Moderator Chatbot for Deliberative Discussion: Effects of Discussion Structure and Discussant FacilitationOnline chat functions as a discussion channel for diverse social issues. However, deliberative discussion and consensus-reaching can be difficult in online chats in part because of the lack of structure. To explore the feasibility of a conversational agent that enables deliberative discussion, we designed and developed DebateBot, a chatbot that structures discussion and encourages reticent participants to contribute. We conducted a 2 (discussion structure: unstructured vs. structured) × 2 (discussant facilitation: unfacilitated vs. facilitated) between-subjects experiment (N = 64, 12 groups). Our findings are as follows: (1) Structured discussion positively affects discussion quality by generating diverse opinions within a group and resulting in a high level of perceived deliberative quality. (2) Facilitation drives a high level of opinion alignment between group consensus and independent individual opinions, resulting in authentic consensus reaching. Facilitation also drives more even contribution and a higher level of task cohesion and communication fairness. Our results suggest that a chatbot agent could partially substitute for a human moderator in deliberative discussions.2021SKSoomin Kim et al.Computer-Supported Conversation and CommunicationCSCW
Designing a Conversational Agent for Sexual Assault Survivors: Defining Burden of Self-Disclosure and Envisioning Survivor-Centered SolutionsSexual assault survivors hesitate to disclose their stories to others and even avoid case-reporting because of psychological, social, and cultural reasons. Thus, conversational agents (CAs) have gained much attention as a potential counselor because CAs’ characteristics (e.g., anonymity) could mitigate various difficulties of human-human interaction (HHI). Despite the potentials, it is difficult to design a CA for survivors because various aspects should be considered. Especially, with traditional HCI approaches only (e.g., need-finding and usability tests), designers could easily miss psychological and subjective burdens that survivors feel toward a new system. Hence, while envisioning a burden-free CA for survivors, we agilely designed and implemented an initial prototype CA (NamuBot) with professionals (the police and counselors). We then conducted a qualitative user study to identify and compare burdens caused by the CA vs. humans. Lastly, we codesigned design features that could reduce the CA-bound burdens with 36 participants (19 survivors and 17 professionals). Notably, our findings showed that 17 survivors preferred reporting their case to NamuBot over humans, expressing far less burden. Although CAs could also place burdens on survivors, the burdens could be alleviated by the features that the survivors and professionals designed. Finally, we present design implications and strategies to develop burden-mitigating CAs for survivors.2021HPHyanghee Park et al.Seoul National UniversityConversational ChatbotsAgent Personality & AnthropomorphismCHI
Human-AI Interaction in Human Resource Management: Understanding Why Employees Resist Algorithmic Evaluation at Workplaces and How to Mitigate BurdensRecently, Artificial Intelligence (AI) has been used to enable efficient decision-making in managerial and organizational contexts, ranging from employment to dismissal. However, to avoid employees’ antipathy toward AI, it is important to understand what aspects of AI employees like and/or dislike. In this paper, we aim to identify how employees perceive current human resource (HR) teams and future algorithmic management. Specifically, we explored what factors negatively influence employees’ perceptions of AI making work performance evaluations. Through in-depth interviews with 21 workers, we found that 1) employees feel six types of burdens (i.e., emotional, mental, bias, manipulation, privacy, and social) toward AI’s introduction to human resource management (HRM), and that 2) these burdens could be mitigated by incorporating transparency, interpretability, and human intervention to algorithmic decision-making. Based on our findings, we present design efforts to alleviate employees’ burdens. To leverage AI for HRM in fair and trustworthy ways, we call for the HCI community to design human-AI collaboration systems with various HR stakeholders.2021HPHyanghee Park et al.Seoul National UniversityExplainable AI (XAI)AI-Assisted Decision-Making & AutomationAlgorithmic Transparency & AuditabilityCHI
Understanding How People Reason about Aesthetic Evaluations of Artificial IntelligenceArtificial intelligence (AI) algorithms are making remarkable achievements even in creative fields such as aesthetics. However, whether those outside the machine learning (ML) community can sufficiently interpret or agree with their results, especially in such highly subjective domains, is being questioned. In this paper, we try to understand how different user communities reason about AI algorithm results in subjective domains. We designed AI Mirror, a research probe that tells users the algorithmically predicted aesthetic scores of photographs. We conducted a user study of the system with 18 participants from three different groups: AI/ML experts, domain experts (photographers), and general public members. They performed tasks consisting of taking photos and reasoning about AI Mirror’s prediction algorithm with think-aloud sessions, surveys, and interviews. The results showed the following: (1) Users understood the AI using their own group-specific expertise; (2) Users employed various strategies to close the gap between their judgments and AI predictions overtime; (3) The difference between users’ thoughts and AI pre-dictions was negatively related with users’ perceptions of the AI’s interpretability and reasonability. We also discuss design considerations for AI-infused systems in subjective domains.2020COChanghoon Oh et al.Explainable AI (XAI)AI-Assisted Creative WritingDIS
TalkingBoogie: Collaborative Mobile AAC System for Non-verbal Children with Developmental Disabilities and Their CaregiversAugmentative and alternative communication (AAC) technologies are widely used to help non-verbal children enable communication. For AAC-aided communication to be successful, caregivers should support children with consistent intervention strategies in various settings. As such, caregivers need to continuously observe and discuss children's AAC usage to create a shared understanding of these strategies. However, caregivers often find it challenging to effectively collaborate with one another due to a lack of family involvement and the unstructured process of collaboration. To address these issues, we present TalkingBoogie, which consists of two mobile apps: TalkingBoogie-AAC for caregiver-child communication, and TalkingBoogie-coach supporting caregiver collaboration. Working together, these applications provide contextualized layouts for symbol arrangement, scaffold the process of sharing and discussing observations, and induce caregivers' balanced participation. A two-week deployment study with four groups (N=11) found that TalkingBoogie helped increase mutual understanding of strategies and encourage balanced participation between caregivers with reduced cognitive loads.2020DSDonghoon Shin et al.Seoul National UniversityAugmentative & Alternative Communication (AAC)Special Education TechnologyCHI
Understanding User Perception of Automated News Generation SystemAutomated journalism refers to the generation of news articles using computer programs. Although it is widely used in practice, its user experience and interface design remain largely unexplored. To understand the user perception of an automated news system, we designed NewsRobot, a research prototype that automatically generated news on major events of the PyeongChang 2018 Winter Olympic Games in real-time. It produces six types of news by combining two kinds of content (general/individualized) and three styles (text, text+image, text+image+sound). A total of 30 users participated in using NewsRobot, completing surveys and interviews on their experience. Our findings are as follows: (1) Users preferred individualized news yet considered it less credible, (2) more presentation elements were appreciated but only if their quality was assured, and (3) NewsRobot was considered factual and accurate yet shallow in depth. Based on our findings, we discuss implications for designing automated journalism user interfaces.2020COChanghoon Oh et al.Carnegie Mellon UniversityGenerative AI (Text, Image, Music, Video)AI-Assisted Decision-Making & AutomationCHI
Bot in the Bunch: Facilitating Group Chat Discussion by Improving Efficiency and Participation with a ChatbotAlthough group chat discussions are prevalent in daily life, they have a number of limitations. When discussing in a group chat, reaching a consensus often takes time, members contribute unevenly to the discussion, and messages are unorganized. Hence, we aimed to explore the feasibility of a facilitator chatbot agent to improve group chat discussions. We conducted a needfinding survey to identify key features for a facilitator chatbot. We then implemented GroupfeedBot, a chatbot agent that could facilitate group discussions by managing the discussion time, encouraging members to participate evenly, and organizing members' opinions. To evaluate GroupfeedBot, we performed preliminary user studies that varied for diverse tasks and different group sizes. We found that the group with GroupfeedBot appeared to exhibit more diversity in opinions even though there were no differences in output quality and message quantity. On the other hand, GroupfeedBot promoted members' even participation and effective communication for the medium-sized group.2020SKSoomin Kim et al.Seoul National UniversityConversational ChatbotsAgent Personality & AnthropomorphismCHI
Comparing Data from Chatbot and Web Surveys: Effects of Platform and Conversational Style on Survey Response QualityThis study aims to explore the feasibility of a text-based virtual agent as a new survey method to overcome the web survey's common response quality problems, which are caused by respondents' inattention. To this end, we conducted a 2 (platform: web vs. chatbot) × 2 (conversational style: formal vs. casual) experiment. We used satisficing theory to compare the responses' data quality. We found that the participants in the chatbot survey, as compared to those in the web survey, were more likely to produce differentiated responses and were less likely to satisfice; the chatbot survey thus resulted in higher-quality data. Moreover, when a casual conversational style is used, the participants were less likely to satisfice–although such effects were only found in the chatbot condition. These results imply that conversational interactivity occurs when a chat interface is accompanied by messages with effective tone. Based on an analysis of the qualitative responses, we also showed that a chatbot could perform part of a human interviewer's role by applying effective communication strategies.2019SKSoomin Kim et al.Seoul National UniversityConversational ChatbotsUser Research Methods (Interviews, Surveys, Observation)CHI