Fork It: Supporting Stateful Alternatives in Computational Notebooks

Document Title

Fork It: Supporting Stateful Alternatives in Computational Notebooks

Document Information

• Subject Area: Data Science Tools and Human-Computer Interaction
• Keywords: Data Science, Computational Notebooks, State Branching, Exploratory Programming, Program History, Decision Points

Research Background and Problem

• Identified Problems or Challenges:
  • Current computational notebooks (e.g., Jupyter Notebook), while popular among data scientists, have functional and design limitations that make it difficult for users to handle multi-state issues in exploratory tasks.
  • Notebooks provide a single execution state, where variable updates may lead to irreversible data loss, recoverable only by re-executing the code.
  • Exploring different decision points often requires repetitive and complex code organization, and the linear layout is often unsuitable for comparing different algorithms, parameters, or data transformations.
• Why This Problem Matters:
  • Exploration is central to data science, but the current state of computational notebooks hinders quick attempts at alternative paths and clear expression of decision points. These limitations affect the efficiency of data analysis and communication of results.
• Research Motivation and Related Work:
  • Existing research focuses on version control or content cleanup for notebooks, but has not deeply explored execution state management.
  • Through a series of interviews, the authors found that data scientists urgently need tools to express different paths, simplifying exploration and comparison.
  • Inspired by nonlinear code presentation tools (e.g., Juxtapose, Code Bubbles) and combining insights from data science workflows, the authors propose directions for improvement.

Solution

• Proposed Method or Solution:
  • Branching (Forking): Create new interpreter sessions (branches) from the current or any past execution state to explore different paths.
  • Backtracking: Browse and return to previous execution states, creating new branches to address issues of "regret" or irreversible state modifications.
  • Visualization: Adopt a multi-path (side-by-side) layout in notebooks, allowing users to compare different branch paths and their results.
• Innovations:
  • Provide core functionality to manage multiple execution states, enabling users to naturally explore various alternatives.
  • Break the linear model limitations of computational notebooks, offering intuitive tools for exploratory programming.
• Implementation Steps and Key Technologies:
  • Develop a Jupyter Notebook extension using Python's serialization library (dill) to save and load execution states.
  • Introduce navigation buttons and side-by-side layouts to visualize code context and variable states, enhancing user interaction with paths.
  • Restrict the creation of one branch at a time while allowing unlimited path additions to optimize simplicity.
  • Support "Restart & Run All" to ensure consistent branch states and reproducible results.

Research Outcomes

• Specific Results:
  • The prototype tool effectively supports data scientists in expressing and comparing paths for exploratory tasks.
  • Users can clearly manage different hypothetical paths without polluting the main code, enabling complex data transformations.
  • The backtracking feature helps some participants efficiently recover from erroneous states without tedious manual restoration.
• Advantages Compared to Existing Solutions:
  • The branching functionality is better suited for capturing decision points in exploratory processes than linear notebooks.
  • Provides support for multiple execution states, not just static code modification-based version control.
  • Enhances user clarity and efficiency in handling complex workflows and organizing code.
• Experiment or Evaluation Results:
  • User testing with 11 professional data scientists showed the tool's effectiveness in exploration, result comparison, and expressiveness.
  • Branching supports various use cases, including model comparison, feature selection, and debugging temporary code snippets.
  • Backtracking usage was limited by time and guidance, but participants acknowledged its potential.
• Limitations and Future Directions:
  • Current tool performance optimization for large-scale data is limited (e.g., saving entire execution states may cause memory issues), requiring further research into compression or incremental storage techniques.
  • Multi-level branching or nested path functionality is not implemented, restricting the organization of complex tasks.
  • Narrow view width affects user experience; improvements could include full-screen mode or enhanced path hiding/switching functionality.
  • Providing intuitive displays of variable states across different paths or supporting automatic linking of shared code between paths could improve expression efficiency.

Conclusion

The branching and backtracking tools proposed in this study effectively extend the linear model of computational notebooks, addressing the practical needs of exploratory tasks in data science workflows. The results demonstrate that these tools significantly enhance the expression of decision points, debugging, and cross-path comparison. This research provides valuable inspiration and design guidelines for future exploration of more complex nonlinear programming support tools.