AI as a Bridge Across Ages: Exploring The Opportunities of Artificial Intelligence in Supporting Inter-Generational Communication in Virtual RealityInter-generational communication is essential for bridging generational gaps and fostering mutual understanding. However, maintaining it is complex due to cultural, communicative, and geographical differences. Recent research indicated that while Virtual Reality (VR) creates a relaxed atmosphere and promotes companionship, it inadequately addresses the complexities of inter-generational dialogue, including variations in values and relational dynamics. To address this gap, we explored the opportunities of Artificial Intelligence (AI) in supporting inter-generational communication in VR. We developed three technology probes (e.g., Content Generator, Communication Facilitator, and Info Assistant) in VR and employed them in a probe-based participatory design study with twelve inter-generational pairs. Our results show that AI-powered VR facilitates inter-generational communication by enhancing mutual understanding, fostering conversation fluency, and promoting active participation. We also introduce several chall2025QDQiuxin Du et al.Caring at a DistanceCSCW
Designing LLM-Powered Multimodal Instructions to Support Rich Hands-on Skills Remote Learning: A Case Study with Massage Instructors and LearnersAlthough remote learning is widely used for delivering and capturing knowledge, it has limitations in teaching hands-on skills that require nuanced instructions and demonstrations of precise actions, such as massage. Furthermore, scheduling conflicts between instructors and learners often limit the availability of real-time feedback, reducing learning efficiency. To address these challenges, we developed a synthesis tool utilizing an LLM-powered Virtual Teaching Assistant (VTA). This tool integrates multimodal instructions that convey precise data, such as stroke patterns and pressure control, while providing real-time feedback for learners and summarizing their performance for instructors. Our case study with instructors and learners demonstrated the effectiveness of these multimodal instructions and the VTA in enhancing massage teaching and learning. We then discuss the tools' use in other hands-on skills instruction and cognitive process differences in various courses.2025CJChutian Jiang et al.Computational Media and Arts Thrust, The Hong Kong University of Science and Technology (Guangzhou)Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Full-Body Interaction & Embodied InputHuman-LLM CollaborationCHI
Bridging the Literacy Gap for Adults: Streaming and Engaging in Adult Literacy Education through LivestreamingLiteracy---the ability to read, write, and comprehend text---is an important topic addressed by UNESCO. Despite global efforts to promote adult literacy education, rural areas with limited resources still lag behind. As livestreaming has gained popularity in China, many streamers leveraged its accessibility and affordability to reach low-literate adults. To gain a better understanding of the practices and challenges faced by adult literacy education through livestreaming, we conducted a mixed-methods study involving a 7-day observation of livestreaming sessions and an interview study with twelve streamers and ten viewers. We discovered streamers' altruistic motives and unique interactive approaches. Viewers perceived livestreaming as a more engaging, community-supportive method than traditional approaches. We also identified both shared and unique challenges for streamers and viewers that limit its efficacy as a learning tool. Finally, we recognized opportunities to enhance educational equity, emphasizing design implications for advancing adult literacy education and promoting diversity in livestreaming.2024SFShihan Fu et al.The Hong Kong University of Science and Technology (Guangzhou)Online Learning & MOOC PlatformsLive Streaming & Content CreatorsCHI
Mapping Accessibility Assignments into Core Computer Science Topics: An Empirical Study with Interviews and Surveys of Instructors and StudentsIncorporating accessibility education into undergraduate computer science (CS) programs is essential for preparing future technology professionals to create inclusive technology. However, many CS programs lack accessibility coverage, often confining it to human-computer interaction (HCI) courses. To address this gap, we developed accessibility assignments seamlessly integrated into core CS courses. We collaborated closely with ten instructors to select and customize these assignments to suit their needs. To evaluate the impact of these assignments, we conducted interviews with instructors and administered surveys and interviews with their students. Our findings indicate significant improvement in students' familiarity with accessibility concepts and confidence in implementation following completion of the assignments. However, their mindset and future interest in accessibility remained the same. Instructors found it straightforward to incorporate these assignments without compromising core computing concepts. In sum, we validated a foundation for effectively resourcing instructors with accessibility teaching materials and increasing their capacity in accessibility knowledge.2024EKEmily Kuang et al.Rochester Institute of TechnologyCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Universal & Inclusive DesignSpecial Education TechnologyCHI
Enhancing UX Evaluation Through Collaboration with Conversational AI Assistants: Effects of Proactive Dialogue and TimingUsability testing is vital for enhancing the user experience (UX) of interactive systems. However, analyzing test videos is complex and resource-intensive. Recent AI advancements have spurred exploration into human-AI collaboration for UX analysis, particularly through natural language. Unlike user-initiated dialogue, our study investigated the potential of proactive conversational assistants to aid UX evaluators through automatic suggestions at three distinct times: before, in sync with, and after potential usability problems. We conducted a hybrid Wizard-of-Oz study involving 24 UX evaluators, using ChatGPT to generate automatic problem suggestions and a human actor to respond to impromptu questions. While timing did not significantly impact analytic performance, suggestions appearing after potential problems were preferred, enhancing trust and efficiency. Participants found the automatic suggestions useful, but they collectively identified more than twice as many problems, underscoring the irreplaceable role of human expertise. Our findings also offer insights into future human-AI collaborative tools for UX evaluation.2024EKEmily Kuang et al.Rochester Institute of TechnologyHuman-LLM CollaborationPrototyping & User TestingComputational Methods in HCICHI
Designing Unobtrusive Modulated Electrotactile Feedback on Fingertip Edge to Assist Blind and Low Vision (BLV) People in Comprehending ChartsCharts are crucial in conveying information across various fields but are inaccessible to blind and low vision (BLV) people without assistive technology. Chart comprehension tools leveraging haptic feedback have been used widely but are often bulky, expensive, and static, rendering them inefficient for conveying chart data. To increase device portability, enable multitasking, and provide efficient assistance in chart comprehension, we introduce a novel system that delivers unobtrusive modulated electrotactile feedback directly to the fingertip edge. Our three-part study with twelve participants confirmed the effectiveness of this system, demonstrating that electrotactile feedback, when applied for 0.5 seconds with a 0.12-second interval, provides the most accurate position and direction recognition. Furthermore, our electrotactile device has proven valuable in assisting BLV participants in comprehending four commonly used charts: line charts, scatterplots, bar charts, and pie charts. We also delve into the implications of our findings on recognition enhancement, presentation modes, and function synergy.2024CJChutian Jiang et al.Computational Media and Arts ThrustVibrotactile Feedback & Skin StimulationVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)CHI
Exploring the Opportunity of Augmented Reality (AR) in Supporting Older Adults to Explore and Learn Smartphone ApplicationsThe global aging trend compels older adults to navigate the evolving digital landscape, presenting a substantial challenge in mastering smartphone applications. While Augmented Reality (AR) holds promise for enhancing learning and user experience, its role in aiding older adults' smartphone app exploration remains insufficiently explored. Therefore, we conducted a two-phase study: (1) a workshop with 18 older adults to identify app exploration challenges and potential AR interventions, and (2) tech-probe participatory design sessions with 15 participants to co-create AR support tools. Our research highlights AR's effectiveness in reducing physical and cognitive strain among older adults during app exploration, especially during multi-app usage and the trial-and-error learning process. We also examined their interactional experiences with AR, yielding design considerations on tailoring AR tools for smartphone app exploration. Ultimately, our study unveils the prospective landscape of AR in supporting the older demographic, both presently and in future scenarios.2024XJXiaofu Jin et al.The Hong Kong University of Science and TechnologyAR Navigation & Context AwarenessAging-Friendly Technology DesignSmart Home Interaction DesignCHI
Enhancing Older Adults' Gesture Typing Experience Using the T9 Keyboard on Small Touchscreen DevicesOlder adults increasingly adopt small-screen devices, but limited motor dexterity hinders their ability to type effectively. While a 9-key (T9) keyboard allocates larger space to each key, it is shared by multiple consecutive letters. Consequently, users must interrupt their gestures when typing consecutive letters, leading to inefficiencies and poor user experience. Thus, we proposed a novel keyboard that leverages the currently unused key 1 to duplicate letters from the previous key, allowing the entry of consecutive letters without interruptions. A user study with 12 older adults showed that it significantly outperformed the T9 with wiggle gesture in typing speed, KSPC, insertion errors, and deletes per word while achieving comparable performance as the conventional T9. Repeating the typing tasks with 12 young adults found that the advantages of the novel T9 were consistent or enhanced. We also provide error analysis and design considerations for improving gesture typing on T9 for older adults.2023EKEmily Kuang et al.Rochester Institute of TechnologyMotor Impairment Assistive Input TechnologiesAging-Friendly Technology DesignUniversal & Inclusive DesignCHI
Bridging the Generational Gap: Exploring How Virtual Reality Supports Remote Communication Between Grandparents and GrandchildrenWhen living apart, grandparents and grandchildren often use audio-visual communication approaches to stay connected. However, these approaches seldom provide sufficient companionship and intimacy due to a lack of co-presence and spatial interaction, which can be fulfilled by immersive virtual reality (VR). To understand how grandparents and grandchildren might leverage VR to facilitate their remote communication and better inform future design, we conducted a user-centered participatory design study with twelve pairs of grandparents and grandchildren. Results show that VR affords casual and equal communication by reducing the generational gap, and promotes conversation by offering shared activities as bridges for connection. Participants preferred resemblant appearances on avatars for conveying well-being but created ideal selves for gaining playfulness. Based on the results, we contribute eight design implications that inform future VR-based grandparent-grandchild communications.2023XWXiaoying Wei et al.IIP(Computational Media and Arts)Social & Collaborative VRIdentity & Avatars in XRCHI
Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text)AI is promising in assisting UX evaluators with analyzing usability tests, but its judgments are typically presented as non-interactive visualizations. Evaluators may have questions about test recordings, but have no way of asking them. Interactive conversational assistants provide a Q&A dynamic that may improve analysis efficiency and evaluator autonomy. To understand the full range of analysis-related questions, we conducted a Wizard-of-Oz design probe study with 20 participants who interacted with simulated AI assistants via text or voice. We found that participants asked for five categories of information: user actions, user mental model, help from the AI assistant, product and task information, and user demographics. Those who used the text assistant asked more questions, but the question lengths were similar. The text assistant was perceived as significantly more efficient, but both were rated equally in satisfaction and trust. We also provide design considerations for future conversational AI assistants for UX evaluation.2023EKEmily Kuang et al.Rochester Institute of TechnologyConversational ChatbotsHuman-LLM CollaborationAI-Assisted Decision-Making & AutomationCHI
"Merging Results Is No Easy Task": An International Survey Study of Collaborative Data Analysis Practices Among UX PractitionersAnalysis is a key part of usability testing where UX practitioners seek to identify usability problems and generate redesign suggestions. Although previous research reported how analysis was conducted, the findings were typically focused on individual analysis or based on a small number of professionals in specific geographic regions. We conducted an online international survey of 279 UX practitioners on their practices and challenges while collaborating during data analysis. We found that UX practitioners were often under time pressure to conduct analysis and adopted three modes of collaboration: independently analyze different portions of the data and then collaborate, collaboratively analyze the session with little or no independent analysis, and independently analyze the same set of data and then collaborate. Moreover, most encountered challenges related to lack of resources, disagreements with colleagues regarding usability problems, and difficulty merging analysis from multiple practitioners. We discuss design implications to better support collaborative data analysis.2022EKEmily Kuang et al.Rochester Institute of TechnologyUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingComputational Methods in HCICHI
"Too old to bank digitally?": A Survey of Banking Practices and Challenges Among Older Adults in ChinaThe banking industry has been integrating digital technologies globally. However, accepting new technologies is challenging in particular for older adults. We focus on older adults' banking experiences in China, where digital transactions have been growing rapidly, to provide a perspective on how they adapt to this trend. We conducted an online survey with 155 older adults who are 60 or above (M=70, SD=9) from 18 provinces to explore their banking practices and challenges. Our results show that older adults conduct banking transactions frequently. However, few do so using digital platforms despite long wait times in physical banks. The main concerns reported by them are about security and usability. Nonetheless, they hold a positive attitude towards digital platforms (e.g., apps, virtual banks). Interestingly, age and gender have significant effects on particular banking behaviors. We discuss our findings in the context of prior studies and highlight design opportunities for improving banking accessibility for older adults.2021XJXiaofu Jin et al.Aging-Friendly Technology DesignPrivacy by Design & User ControlDIS