SwipeSense: Exploring the Feasibility of Back-of-Device Swipe Interaction Using Built-In IMU SensorsThe growing dimensions of smartphones have intensified the challenges associated with screen reachability. Back-of-device (BoD) interaction expands the range of reachability and offers a promising solution to mitigate screen occlusion while enhancing one-handed interactions. However, much of the existing research relies on incorporating additional hardware components. In this paper, we present SwipeSense a technique for exploring the feasibility of directional swipe interactions on the back of devices, utilizing built-in inertial measurement unit (IMU) sensors and machine learning models. We conducted a user study with 12 participants who performed 9600 BoD swipes in 8 distinct directions while holding the device naturally. The results of our machine learning models indicate that various directional swipes on the back of the device can be accurately distinguished using only the built-in IMU sensors of the phone, achieving a range of model accuracy between 72% and 95%. Furthermore, we showcase potential applications for these gestures.2025NSNeel Shah et al.Hand Gesture RecognitionContext-Aware ComputingUbiquitous ComputingMobileHCI
Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image ColorizationColor design is essential in areas such as product, graphic, and fashion design. However, current tools like Photoshop, with their concrete-driven color manipulation approach, often stumble during early ideation, favoring polished end results over initial exploration. We introduced Mondrian as a test-bed for abstraction-driven approach using interactive color palettes for image colorization. Through a formative study with six design experts, we selected three design options for visual abstractions in color design and developed Mondrian where humans work with abstractions and AI manages the concrete aspects. We carried out a user study to understand the benefits and challenges of each abstraction format and compare the Mondrian with Photoshop. A survey involving 100 participants further examined the influence of each abstraction format on color composition perceptions. Findings suggest that interactive visual abstractions encourage a non-linear exploration workflow and an open mindset during ideation, thus providing better creative affordance.2024XSXinyu Shi et al.University of WaterlooGenerative AI (Text, Image, Music, Video)Graphic Design & Typography ToolsCreative Collaboration & Feedback SystemsCHI
Escapement: A Tool for Interactive Prototyping with Video via Sensor-Mediated Abstraction of Time We present Escapement, a video prototyping tool that introduces a powerful new concept for prototyping screen-based interfaces by flexibly mapping sensor values to dynamic playback control of videos. This recasts the time dimension of video mock-ups as sensor-mediated interaction. This abstraction of time as interaction, which we dub video-escapement prototyping, empowers designers to rapidly explore and viscerally experience direct touch or sensor-mediated interactions across one or more device displays. Our system affords cross-device and bidirectional remote (tele-present) experiences via cloud-based state sharing across multiple devices. This makes Escapement especially potent for exploring multi-device, dual-screen, or remote-work interactions for screen-based applications. We introduce the core concept of sensor-mediated abstraction of time for quickly generating video-based interactive prototypes of screen-based applications, share the results of observations of long-term usage of video-escapement techniques with experienced interaction designers, and articulate design choices for supporting a reflective, iterative, and open-ended creative design process.2023MNMolly Jane Nicholas et al.UC BerkeleyTeleoperation & TelepresencePrototyping & User TestingCHI
Similarity-Based Explanations meet Matrix Factorization via Structure-Preserving EmbeddingsEmbeddings are core components of modern model-based Collaborative Filtering (CF) methods, such as Matrix Factorization (MF) and Deep Learning variations. In essence, embeddings are mappings of the original sparse representation of categorical features (e.g., user and items) to dense low-dimensional representations. A well-known limitation of such methods is that the learned embeddings are opaque and hard to explain to the users. On the other hand, a key feature of simpler KNN-based CF models (aka user/item-based CF) is that they naturally yield similarity-based explanations, i.e., similar users/items as evidence to support model predictions. Unlike related works that try to attribute explicit meaning to the learned embeddings, in this paper, we propose to equip the learned embeddings of MF with meaningful similarity-based explanations. First, we show that the learned user/item embeddings of MF do not preserve the distances between users (or items) in the original rating data. This may prevent meaningful similarity-based explanations. Next, we propose a novel approach that initializes SGD with user/item embeddings that preserve the structural properties of the original input data. We conduct a broad set of experiments and show that our method enables explanations, very similar to the ones provided by KNN-based approaches, without harming the prediction performance. Moreover, we show that fine-tuning the structure-preserving embeddings may unlock better local minima in the optimization space, leading simple vanilla MF to reach competitive performances with the best-known models for the rating prediction task2022LMLeandro Balby Marinho et al.Explainable AI (XAI)Recommender System UXIUI
TiiS: Learn, Generate, Rank, Explain: A Case Study of Visual Explanation by Generative Machine LearningKim 等人提出了一个整合学习、生成、排序与解释功能的视觉解释系统，通过生成式机器学习为用户提供可解释的视觉内容分析。2022CKChris Kim et al.Explainable AI (XAI)AI-Assisted Creative WritingIUI
Lexichrome: Text Construction and Lexical Discovery with Word-Color Associations Using Interactive VisualizationBased on word-color associations from a comprehensive, crowdsourced lexicon, we present Lexichrome: a web application that explores the popular perception of relationships between English words and eleven basic color terms using interactive visualization. Lexichrome provides three complementary visualizations: "Palette" presents the diversity of word-color associations across the color palette; "Words" reveals the color associations of individual words using a dictionary-like interface; "Roget's Thesaurus" uncovers color association patterns in different semantic categories found in the thesaurus. Finally, our text editor allows users to compose their own texts and examine the resultant chromatic fingerprints throughout the process. We studied the utility of Lexichrome in a two-part qualitative user study with nine participants from various writing-intensive professions. We find that the presence of word-color associations promotes awareness surrounding word choice, editorial decision, and audience reception, and introduce a variety of use cases, features, and opportunities applicable to creative writing, corporate communication, and journalism.2020CKChris Kim et al.Interactive Data VisualizationData StorytellingDIS
Aggregated Visualization of Playtesting DataPlaytesting is a key component in the game development process aimed at improving the quality of games through the collection of gameplay data and identification of design issues. Visualization techniques are currently being employed to help integrate quantitative and qualitative data. Despite that, two existing challenges are to determine the level of detail to be presented to developers based on their needs and to effectively communicate the collected data so that informed design changes can be reached. In this paper, we first propose an aggregated visualization technique that makes use of clustering, territory tessellation, and trajectory aggregation to simultaneously display mixed playtesting data. Secondly, to assess the usefulness of our technique we evaluate it through interviews with professional game developers and compare it to a non-aggregated visualization. The results of this study also provide an important contribution towards identifying areas of improvement in the portrayal of gameplay data.2019GWGünter Wallner et al.Eindhoven University of TechnologyInteractive Data VisualizationGame UX & Player BehaviorCHI
Let's Play Together: Adaptation Guidelines of Board Games for Players with Visual ImpairmentBoard games present accessibility barriers for players with visual impairment since they often employ visuals alone to communicate gameplay information. Our research focuses on board game accessibility for those with visual impairment. This paper describes a three-phase study conducted to develop board game accessibility adaptation guidelines. These guidelines were developed through a user-centered design approach that included in-depth interviews and a series of user studies using two adapted board games. Our findings indicate that participants with and without visual impairment were able to play the adapted games, exhibiting a balanced experience whereby participants had complete autonomy and were provided with equal chances of victory. Our paper also contributes to the game and accessibility communities through the development of adaptation guidelines that allow board games to become inclusive irrespective of a player's visual impairment.2019FFFrederico da Rocha Tomé Filho et al.University of Ontario Institute of TechnologyUniversal & Inclusive DesignGame AccessibilityCHI
Saliency Deficit and Motion Outlier Detection in Animated ScatterplotsWe report the results of a crowdsourced experiment that measured the accuracy of motion outlier detection in multivariate, animated scatterplots. The targets were outliers either in speed or direction of motion, and were presented with varying levels of saliency in dimensions that are irrelevant to the task of motion outlier detection (e.g., color, size, position). We found that participants had trouble finding the outlier when it lacked irrelevant salient features and that visual channels contribute unevenly to the odds of an outlier being correctly detected. Direction of motion contributes the most to accurate detection of speed outliers, and position contributes the most to accurate detection of direction outliers. We introduce the concept of saliency deficit in which item importance in the data space is not reflected in the visualization due to a lack of saliency. We conclude that motion outlier detection is not well supported in multivariate animated scatterplots.2019RVRafael Veras et al.University of Ontario Institute of TechnologyInteractive Data VisualizationVisualization Perception & CognitionCHI
Games and Play SIG: Engaging Small Developer CommunitiesThe Games-and-Play community has thrived at ACM SIGCHI with a consistent increase in games- and play-related submissions across research papers, workshops, posters, demos, and competitions. The community has attracted a significant number of academic researchers, students, and practitioners to CHI conferences in recent years. CHI 2018 is being held in Montréal, a major game development hub. Montréal is not only a home for major game studios but also more than 100 smaller game studios. In line with the “Engage With CHI” spirit of CHI 2018, this SIG aims to engage the Games and Play community in a discussion about the directions that we can take to advance towards demographics that will benefit from HCI games research but are currently underrepresented: small, independent developers, non-profit organizations, and academics that create mobile games, games for health, games for change, and/or educational games.2018LNLennart E. Nacke et al.University of WaterlooGame UX & Player BehaviorSerious & Functional GamesCollaborative Learning & Peer TeachingCHI
Metatation: Annotation as Implicit Interaction to Bridge Close and Distant ReadingIn the domain of literary criticism, many critics practice close reading, annotating by hand while performing a detailed analysis of a single text. Often this process employs the use of external resources to aid analysis. In this article, we present a study and subsequent tool design focused on leveraging a critic’s annotations as implicit interactions for initiating context-specific computational support that automatically searches external resources. We observed 14 poetry critics performing a close reading, revealing a set of cognitive practices supported through free-form annotation that have not previously been discussed in this context. We used guidelines derived from our study to design a tool, Metatation, which uses a pen-and-paper system with a peripheral display to utilize reader annotations as underspecified interactions to augment close reading. By turning paper based annotations into implicit queries, Metatation provides relevant supplemental information in a just-in-time manner and acts as a bridge between close and distant reading.2018HMHrim R Mehta et al.University of Ontario Institute of TechnologyUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI