HyPockeTuner: Bringing Hyperparameter Optimization to Mobile DevicesHyperparameter optimization (HPO) is a long-running process that can span hours or even days. While recent Human-in-the-Loop HPO systems enable monitoring and steering of the process, they are typically designed for desktop environments, which limits their effectiveness in managing prolonged experiments in practice. To address these limitations, we present HyPockeTuner, an interactive mobile system that enables users to monitor, steer, and reflect on HPO experiments anytime, anywhere from smartphones. Its mobile-tailored interface supports tracking experiment history and visualizing the relationship between user interventions and performance changes. HyPockeTuner also employs a notification workflow that alerts users to important events, reducing the burden of constant monitoring while enabling timely interventions. In a pilot study, we validated that users could readily identify critical events, such as performance improvements and intervention points, through our visualization. Furthermore, two five-day deployment studies with follow-up reflection sessions demonstrated that users could integrate experiment management into their daily routines and reflect on past decisions, generating insights for future improvement.2026DHDonghee Hong et al.Sungkyunkwan UniversityAutoML InterfacesRemote Work Tools & ExperienceBehavior Change & Reflection TechnologyCHI
CrossLit: Connecting Visual and Textual Sensemaking for Literature ReviewConducting literature reviews is cognitively demanding, requiring researchers to navigate large volumes of work while constructing coherent narratives that position their contributions. The process unfolds through iterative stages of sensemaking, each demanding different support. Existing tools emphasize either visual interfaces that provide macroscopic overviews or textual interfaces that support thematic organization and narrative construction. However, keeping modalities separate forces researchers to switch between tools, disrupting workflow continuity. We present CrossLit, a system that integrates and synchronizes visual and textual interfaces to support the entire process from discovering papers to composing coherent narratives. CrossLit allows researchers to group and annotate papers visually while generating aligned textual structures, and to edit text that automatically updates visual representations. We find that CrossLit helps users develop and refine conceptual structures and build narratives iteratively through seamless cross-modal transitions. We conclude by discussing design implications for synchronizing visual and textual interfaces for sensemaking support.2026KCKiroong Choe et al.Seoul National UniversityInteractive Data VisualizationCollaborative Writing ToolsAnnotation & Markup ToolsCHI
Bridging Gulfs in UI Generation through Semantic GuidanceWhile generative AI enables high-fidelity UI generation from text prompts, users struggle to articulate design intent and evaluate or refine results—creating gulfs of execution and evaluation. To understand the information needed for UI generation, we conducted a thematic analysis of UI prompting guidelines, identifying key design semantics and discovering that they are hierarchical and interdependent. Leveraging these findings, we developed a system that enables users to specify semantics, visualize relationships, and extract how semantics are reflected in generated UIs. By making semantics serve as an intermediate representation between human intent and AI output, our system bridges both gulfs by making requirements explicit and outcomes interpretable. A comparative user study suggests that our approach enhances users' perceived control over intent expression and outcome interpretation, and facilitates more predictable iterative refinement. Our work demonstrates how explicit semantic representation enables systematic and explainable exploration of design possibilities in AI-driven UI design.2026SPSeokhyeon Park et al.Seoul National UniversityGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationExplainable AI (XAI)CHI
Good Fences Make Good Learning: How Self-Directed Language Learners Navigate LLM Delegation DecisionsSelf-directed language learners increasingly turn to large language models (LLMs) for assistance, but face the challenge of deciding what learning tasks to delegate to LLMs and how. While prior research has examined the effectiveness of LLM in improving language proficiency, less is known about how learners negotiate agency and what values guide delegation strategies. To address this gap, we conducted a two-part study: an analysis of discussions in the r/languagelearning subreddit to map learners' LLM usage patterns and factors driving delegation, followed by a technology probe study where learners designed learning activities and experimented with LLM support. Our findings reveal three key considerations influencing delegation: accuracy, independence, and authenticity. We analyze these considerations through two types of obstacles: selection challenges in choosing appropriate strategies and execution challenges in following through on intentions. These insights inform the design of AI-assisted learning systems that preserve learner agency while supporting diverse learning goals.2026JSJiwon Song et al.Seoul National UniversityHuman-LLM CollaborationIntelligent Tutoring Systems & Learning AnalyticsCHI
Unveiling High-dimensional Backstage: A Survey for Reliable Visual Analytics with Dimensionality ReductionDimensionality reduction (DR) techniques are essential for visually analyzing high-dimensional data. However, visual analytics using DR often face unreliability, stemming from factors such as inherent distortions in DR projections. This unreliability can lead to analytic insights that misrepresent the underlying data, potentially resulting in misguided decisions. To tackle these reliability challenges, we review 133 papers that address the unreliability of visual analytics using DR. Through this review, we contribute (1) a workflow model that describes the interaction between analysts and machines in visual analytics using DR, and (2) a taxonomy that identifies where and why reliability issues arise within the workflow, along with existing solutions for addressing them. Our review reveals ongoing challenges in the field, whose significance and urgency are validated by five expert researchers. This review also finds that the current research landscape is skewed toward developing new DR techniques rather than their interpretation or evaluation, where we discuss how the HCI community can contribute to broadening this focus.2025HJHyeon Jeon et al.Seoul National University, Department of Computer Science and EngineeringInteractive Data VisualizationUncertainty VisualizationVisualization Perception & CognitionCHI
Leveraging Multimodal LLM for Inspirational User Interface SearchInspirational search, the process of exploring designs to inform and inspire new creative work, is pivotal in mobile user interface (UI) design. However, exploring the vast space of UI references remains a challenge. Existing AI-based UI search methods often miss crucial semantics like target users or the mood of apps. Additionally, these models typically require metadata like view hierarchies, limiting their practical use. We used a multimodal large language model (MLLM) to extract and interpret semantics from mobile UI images. We identified key UI semantics through a formative study and developed a semantic-based UI search system. Through computational and human evaluations, we demonstrate that our approach significantly outperforms existing UI retrieval methods, offering UI designers a more enriched and contextually relevant search experience. We enhance the understanding of mobile UI design semantics and highlight MLLMs' potential in inspirational search, providing a rich dataset of UI semantics for future studies.2025SPSeokhyeon Park et al.Seoul National University, Department of Computer Science and EngineeringHuman-LLM CollaborationInteractive Data VisualizationCHI
CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language ModelsLarge language models (LLMs) have facilitated significant strides in generating conversational agents, enabling seamless, contextually relevant dialogues across diverse topics. However, the existing LLM-driven conversational agents have fixed personalities and functionalities, limiting their adaptability to individual user needs. Creating personalized agent personas with distinct expertise or traits can address this issue. Nonetheless, we lack knowledge of how people customize and interact with agent personas. In this research, we investigated how users customize agent personas and their impact on interaction quality, diversity, and dynamics. To this end, we developed CloChat, an interface supporting easy and accurate customization of agent personas in LLMs. We conducted a study comparing how participants interact with CloChat and ChatGPT. The results indicate that participants formed emotional bonds with the customized agents, engaged in more dynamic dialogues, and showed interest in sustaining interactions. These findings contribute to design implications for future systems with conversational agents using LLMs.2024JHJuhye Ha et al.Graduate School of Information Yonsei University, Graduate School of Information Yonsei UniversityAgent Personality & AnthropomorphismHuman-LLM CollaborationCHI
Natural Language Dataset Generation Framework for Visualizations Powered by Large Language ModelsWe introduce VL2NL, a Large Language Model (LLM) framework that generates rich and diverse NL datasets using Vega-Lite specifications as input, thereby streamlining the development of Natural Language Interfaces (NLIs) for data visualization. To synthesize relevant chart semantics accurately and enhance syntactic diversity in each NL dataset, we leverage 1) a guided discovery incorporated into prompting so that LLMs can steer themselves to create faithful NL datasets in a self-directed manner; 2) a score-based paraphrasing to augment NL syntax along with four language axes. We also present a new collection of 1,981 real-world Vega-Lite specifications that have increased diversity and complexity than existing chart collections. When tested on our chart collection, VL2NL extracted chart semantics and generated L1/L2 captions with 89.4% and 76.0% accuracy, respectively. It also demonstrated generating and paraphrasing utterances and questions with greater diversity compared to the benchmarks. Last, we discuss how our NL datasets and framework can be utilized in real-world scenarios. The codes and chart collection are available at https://github.com/hyungkwonko/chart-llm.2024KKKwon Ko et al.KAISTHuman-LLM CollaborationInteractive Data VisualizationTime-Series & Network Graph VisualizationCHI
DataHalo: A Customizable Notification Visualization System for Personalized and Longitudinal InteractionsPeople struggle with the overflow of smartphone notifications but often face two challenges: (1) prioritizing the informative notifications as they wish and (2) retaining the delivered information as long as they want to utilize it. In this paper, we present DataHalo, a customizable notification visualization system that represents notifications as prolonged ambient visualizations on the home screen. DataHalo supports keyword-based filtering and categorization, and draws graphical marks based on time-varying importance model to enable longitudinal interaction with the notifications. We evaluated DataHalo through a usability study ($N$ = 17), from which we improved the interface. We then conducted a three-week deployment study ($N$ = 12) to assess how people use DataHalo in their domestic contexts. Our study revealed that people generated various visualization settings for different kinds of apps. Drawing on both quantitative and qualitative findings, we discussed implications for supporting effective notification management through customizable ambient visualizations.2023GHGuhyun Han et al.Seoul National UniversityNotification & Interruption ManagementCHI
We-toon: A Communication Support System between Writers and Artists in Collaborative Webtoon Sketch RevisionWe present a communication support system, namely \textit{We-toon}, that can bridge the webtoon writers and artists during sketch revision (i.e., character design and draft revision). In the highly iterative design process between the webtoon writers and artists, writers often have difficulties in precisely articulating their feedback on sketches owing to their lack of drawing proficiency. This drawback makes the writers rely on textual descriptions and reference images found using search engines, leading to indirect and inefficient communications. Inspired by a formative study, we designed \textit{We-toon} to help writers revise webtoon sketches and effectively communicate with artists. Through a GAN-based image synthesis and manipulation, \textit{We-toon} can interactively generate diverse reference images and synthesize them locally on any user-provided image. Our user study with 24 professional webtoon authors demonstrated that \textit{We-toon} outperforms the traditional methods in terms of communication effectiveness and the writers' satisfaction level related to the revised image.2022HKHyung-Kwon Ko et al.Generative AI (Text, Image, Music, Video)Creative Collaboration & Feedback SystemsUIST
Understanding Personal Productivity: How Knowledge Workers Define, Evaluate, and Reflect on Their ProductivityProductivity tracking tools often determine productivity based on the time interacting with work-related applications. To deconstruct productivity's diverse and nebulous nature, we investigate how knowledge workers conceptualize personal productivity and delimit productive tasks in both work and non-work contexts. We report a 2-week diary study followed by a semi-structured interview with 24 knowledge workers. Participants captured productive activities and provided the rationale for why the activities were assessed to be productive. They reported a wide range of productive activities beyond typical desk-bound work–ranging from having a personal conversation with dad to getting a haircut. We found six themes that characterize the productivity assessment–work product, time management, worker's state, attitude toward work, impact & benefit, and compound task–and identified how participants interleaved multiple facets when assessing their productivity. We discuss how these findings could inform the design of a comprehensive productivity tracking system that covers a wide range of productive activities.2019YKYoung-Ho Kim et al.Seoul National UniversityKnowledge Worker Tools & WorkflowsPrototyping & User TestingCHI
Wall-based Space Manipulation Technique for Efficient Placement of Distant Objects in Augmented RealityWe present a wall-based space manipulation (WSM) technique that enables users to efficiently select and move distant objects by dynamically squeezing their surrounding space in augmented reality. Users can bring a target object closer by dragging a solid plane behind the object and squeezing the space between them and the plane so that they can select and move the object more delicately and efficiently. We furthermore discuss the unique design challenges of WSM, including the dimension of space reduction and the recognition of the reduced space in relation to the real space. We conducted a user evaluation to verify how WSM improves the performance of the hand-centered object manipulation technique on the HoloLens for moving near objects far away and vice versa. The results indicate that WSM overall performed consistently well and significantly improved efficiency while alleviating arm fatigue.2018HCHan Joo Chae et al.Full-Body Interaction & Embodied InputAR Navigation & Context AwarenessUIST