Glinda: Supporting Data Science with Live Programming, GUIs and a Domain-specific Language

Title of the Paper

Glinda: Supporting Data Science with Live Programming, GUIs and a Domain-specific Language

Paper Information

• Subject Area: Research on data science tools and interactive programming
• Keywords: Data science, live programming, domain-specific language, graphical user interface, visualization, interactive systems, DSL

Research Background and Problem

• Problems and Challenges:

  • Data scientists express numerous dissatisfactions with computational notebooks (e.g., Jupyter), including issues of "inconsistency" caused by a flexible execution model (e.g., dead code, outdated results, cumbersome code cleanup).
  • The complexity and inefficiency of performing most data analysis tasks using scripts limit productivity and ease of use.
  • Data science tools (e.g., Excel, Tableau) offer the convenience of GUIs but lack the openness and flexibility of programming environments.
  • There is a lack of integrated solutions combining live programming, GUIs, and domain-specific languages (DSLs).

• Significance of the Research:

  • Enhancing the efficiency of data science workflows and improving support for analysis and programming.
  • Providing a unified tool to bridge the gap between existing computational notebooks and data science tools.

• Motivation and Related Work:

  • Previous research has attempted to improve the experience of computational notebooks (e.g., automatic cleanup tools, history visualization), but no groundbreaking innovations have emerged.
  • GUI tools and DSLs have shown potential in improving user-friendliness and abstraction capabilities.
  • Live programming has proven effective in data visualization and user interaction, particularly in addressing execution inconsistency issues.

Solution

• Method or Solution:

  • Proposed and implemented a tool named "Glinda," which integrates live programming, graphical user interfaces (GUIs), and a domain-specific language (DSL) specifically designed for data science tasks.
  • Utilized YAML language to describe high-level, declarative steps, supporting data reading, cleaning, transformation, visualization, and machine learning model building and testing.
  • The interface is built on Visual Studio Code, combining interactive result display and real-time feedback.

• Innovations:

  • The first tool to integrate live programming, GUI-based interaction, and DSL into a single data science tool.
  • Adopted an extensible compilation architecture that dynamically compiles the DSL into Python code, with a "recipe" mechanism enabling modularity and extensibility.
  • Achieved bidirectional synchronization between GUI and script content, balancing interactivity and precise expression.

• Implementation Steps and Key Techniques:

  1. Implemented workflow compilation and modularization through Python functions called "recipes" and YAML templates.
  2. Translated YAML descriptions into Python code, executed in the background via Jupyter Server.
  3. Leveraged live programming algorithms to provide immediate execution and feedback for every user input.
  4. Supported interactive feedback, including data table displays and visualization of model training (e.g., loss and accuracy curves).

Research Results

• Specific Outcomes:

  • Successfully developed a tool that integrates DSL, live programming, and interactive GUIs.
  • Glinda received positive feedback in user surveys, with users particularly appreciating its real-time feedback and efficient workflow definition capabilities.

• Advantages:

  • Compared to traditional computational notebooks, it offers higher programming efficiency, a user-friendly interface, and eliminates the need for manual control of execution order.
  • Combines GUI interaction with DSL scripting, making it both user-friendly and retaining the replicability and flexibility of code.
  • The "recipe" design framework enhances modularity and reusability.

• Experimental and Evaluation Results:

  • Qualitative evaluations from 12 experienced professional data scientists showed overwhelmingly positive attitudes towards Glinda.
  • Survey results indicated that users found Glinda more efficient, flexible, and significantly reduced the time required to learn common API usage compared to traditional notebooks.
  • Limitations were mainly related to performance optimization (e.g., efficiency issues with large datasets) and customization for advanced features (e.g., advanced visualization controls).

• Limitations and Future Directions:

  • Limitations:
    • Real-time programming may encounter performance bottlenecks with large datasets or complex computations.
    • The current DSL has room for improvement in complex expressions, such as variable scoping and flexibility.
    • Lacks comprehensive visualization export and report generation features.
  • Future Directions:
    • Optimize real-time computation performance (e.g., adding pause functionality or sampling strategies).
    • Enhance DSL design to support more complex data science tasks (e.g., graph neural networks).
    • Expand support for other user roles (e.g., program managers and operations engineers) to further promote the democratization of data science.

Conclusion

Glinda demonstrates the powerful potential of combining live programming, GUIs, and DSLs, offering a novel way of working in the field of data science. Future improvements and extensions will continue to focus on enhancing efficiency, usability, and expanding its applicability, thereby having a profound impact on collaboration within data science and software teams.