Closing the Loop between User Stories and GUI Prototypes: An LLM-Based Assistant for Cross-Functional Integration in Software Development

Research Background and Issues

• Identified Problems or Challenges: GUI design is central to software development, but the GUI prototyping work involving UX designers, product managers, and developers often faces challenges such as fragmented tools, synchronization difficulties, and frequent changes in requirements. These issues lead to high coordination costs, delayed prototype updates, and inaccuracies in requirement implementation. Although previous studies have proposed solutions based on natural language processing or large language models (LLMs), these methods lack the ability to tightly integrate user requirements (e.g., user stories) with GUI prototypes and have not been systematically or widely adopted in practice.
• Importance: GUI prototypes are critical tools for communicating requirements, and their quality and efficiency are essential for cross-functional team collaboration in the software development process. Addressing these obstacles can not only improve development efficiency but also enhance product quality and workflow consistency within teams.
• Research Motivation and Related Work:
  • Prior studies mainly focused on tools for generating design suggestions (e.g., systems analyzing design complexity metrics) and frameworks for generating design schemes from natural language, but they failed to effectively integrate user requirement descriptions with GUI components.
  • Product managers and UX designers are open to AI-assisted design, presenting an opportunity to explore how LLMs can efficiently integrate GUI prototypes with user requirements.

Solution

• Method or Solution: The authors propose an LLM-based GUI prototyping assistant, with key innovations including integrated detection of user stories and prototypes, GUI component generation, and automatic generation of related user stories.
• Innovations:
  • Integration of user requirements (e.g., user stories in JIRA systems) into the Figma design tool, eliminating the need to switch between tools.
  • Use of LLMs for detecting, matching, and generating GUI components based on user stories, offering greater efficiency and flexibility compared to typical UI generation models based on training data.
  • A two-stage GUI generation architecture that reduces token usage by 64.35%, lowering costs and improving generation performance.
• Implementation Steps and Key Technologies:
  1. User Story Detection: LLM analyzes the implementation status of GUI components in Figma, enabling automated or manual coverage detection.
  2. Matching User Requirements with GUI Components: By analyzing the structure and attributes of GUI components, the system assists users in locating interface elements corresponding to requirements.
  3. GUI Component Generation: Generates high-precision GUI components in stages based on user stories and design context, automatically placing them into prototypes.
  4. Automatic Derivation of User Stories: Generates corresponding user stories from GUI prototypes, allowing product managers to quickly iterate requirements.

Research Outcomes

• Specific Outcomes:
  • Experiments demonstrate that the LLM-based assistant can generate GUI components that are functionally complete and highly aligned with requirements.
  • Compared to traditional manual design, LLM-supported GUI component generation significantly improves user story completion and task efficiency.
  • The automatic extraction of user stories from GUI prototypes meets high standards of quality, accuracy, and scenario-specific needs.
• Advantages Over Existing Solutions:
  • Direct integration of user story viewing and synchronization within Figma reduces tool-switching and enhances team collaboration efficiency.
  • Automatically generated components seamlessly integrate into existing design workflows, requiring no additional training data or domain-specific knowledge compared to models based on training or fine-tuning.
  • The proposed two-stage model effectively reduces input and generation costs, making it more suitable for large-scale design tasks.
• Experimental or Evaluation Results:
  • Users participating in the experiments showed a preference for using the LLM assistant with generation capabilities (satisfaction increased from 3.00 to 3.75), with significant reductions in design task burden (NASA TLX) and frustration levels.
  • GUI generation tasks achieved an average satisfaction score of 7.5/9 (user story completion rate).
  • Automatically generated user stories demonstrated high accuracy and detail levels (average score of 6.7/9).
• Limitations and Future Directions:
  • Limitations:
    • Current prototype design only supports mobile GUI and primarily relies on the Material Design component library, without extending to other platforms.
    • User stories and GUI generation are limited to functional requirements, lacking support for non-functional requirements (e.g., branding styles).
    • Cross-user role collaboration features, such as version control for user stories or developer annotation support, are not fully implemented.
  • Future Directions:
    • Expand support to other types of user interfaces and component libraries to enhance generation capabilities.
    • Add support for non-functional requirements, such as personalized styles and color options.
    • Optimize recommendation diversity and allow users to manipulate generation instructions for greater flexibility.
    • Conduct in-depth studies on the behavioral patterns of cross-functional teams simultaneously using the assistant during real-world collaboration.

The analysis above demonstrates that this study effectively advances the intelligent integration of GUI prototypes and requirement descriptions, proposing practical workflow improvement tools.