Enhancing Computational Notebooks with Code+Data Space Versioning

Research Background and Issues

What problems or challenges did the authors identify?

• The current design of Jupyter Notebook only supports sequential data exploration and does not accommodate nonlinear exploration methods (e.g., rollback, branching, and full restoration), resulting in inefficiencies for data scientists handling complex exploratory tasks.
• Existing tools (e.g., ForkIt) provide limited support for nonlinear exploration but lack a unified and efficient mechanism, making it difficult to flexibly manage rollbacks or branching.
• Insufficient caching and restoration of kernel states (e.g., variable values and execution history), which are critical for efficient nonlinear exploration.

Why is this issue important?

• Data science workflows often require iterative attempts with multiple models, feature engineering, or data processing pipelines, and current tools are cumbersome in supporting such complex exploratory iterations.
• Existing mechanisms fail to adequately meet the need for quickly reverting to earlier data states, especially for tasks involving computationally intensive operations, which can waste significant time and resources.
• Enhancing tool flexibility and user experience can significantly improve the efficiency and productivity of data science practices.

Research Motivation and Related Work

• While existing tools (e.g., git and other notebook management tools) support code version control, they neglect support for data and kernel states.
• For instance, Verdant provides history browsing, and Loops allows comparison of different code branches, but these tools lack support for "data states."
• ForkIt attempts to support branch management by creating independent kernels for each branch but lacks efficient support for historical rollback.

Solution

What methods or solutions did the authors propose?

• Proposed a two-dimensional code+data space version management mechanism by introducing new primitives: rollback and code+data checkout, to support consistent state navigation.
• Developed a prototype system Kishuboard, which seamlessly integrates with Jupyter and allows users to rollback execution or checkout states within complex exploration histories.

What are the innovative aspects of this solution?

1. Unified management of historical states across both code and data dimensions, differing from existing one-dimensional version control (e.g., git).
2. Supports simultaneous checkout of code and data or rollback of data alone, flexibly meeting diverse exploration needs.
3. Introduced a consistency checking mechanism to ensure the integrity of code and data states.
4. Mapped the two-dimensional history onto a one-dimensional axis, providing an intuitive user interface and enabling efficient navigation.

What are the implementation steps and key technologies used?

1. Two-dimensional version management modeling:
   • Code states consist of notebook cells and outputs.
   • Data states include variable mappings and execution history.
2. User interface design:
   • History graph (commit history graph) provides a one-dimensional linear representation.
   • Variable panel and code view display the contents of each commit point.
   • Two control “knobs” adjust code and data states independently.
3. System architecture:
   • A plugin monitors Jupyter’s state changes and records the state after each code execution.
   • Data storage manages commit history, variable states, and branches.
4. Consistency checking and safe operations:
   • Prevents users from performing actions that could lead to inconsistent states (e.g., checking out code to a future version).
   • Ensures consistency between versions during rollback and code+data checkout operations.

Research Results

What specific results were achieved?

• Developed the prototype system Kishuboard, which deeply integrates with Jupyter to support real-time code+data version management.
• Provided an intuitive one-dimensional history graph interface that simplifies navigation in multi-branch exploration.
• Enabled users to quickly rollback or jump to any past state using rollback and checkout without needing to rerun all code.

How does it compare to existing solutions?

1. Simultaneous management of both code and data states, which many existing tools cannot achieve.
2. Automatic checks and prevention of potentially inconsistent operations, enhancing tool stability and safety.
3. Introduced two key features, "rollback" and "checkout historical data," which significantly simplify nonlinear exploration.

What are the experimental or evaluation results?

• Experiment groups: The authors recruited 20 users to conduct comparative experiments on computationally intensive tasks and lightweight tasks.
• Efficiency improvement:
  • For computationally intensive tasks, users using Kishuboard reduced task completion time by an average of 62%.
  • For lightweight tasks, the improvement was lower (25.61%) due to the minimal time cost of rerunning code.
• User feedback:
  • 95% of users expressed willingness to use Kishuboard in future projects.
  • 65% of users indicated willingness to pay at least $5/month for a subscription to the tool.
• Experience evaluation:
  • Users generally found the tool’s functionality practical, especially the "code+data checkout" and "execution history visualization."
  • The interface design received high scores, though some graphical symbols (e.g., differences between dashed and solid lines) were rated moderately in usability.

Limitations and Future Directions

1. Insufficient collaboration support: The tool currently focuses on individual use and does not address multi-user collaboration needs, such as merging variable states.
2. Semantic search of history records: The tool only supports keyword-based queries of history records and does not enable complex semantic searches (e.g., "state when the variable followed a Gaussian distribution").
3. Limited large-scale comparison capabilities: The system’s comparison functionality is restricted to two versions, and future developments should include automated comparison tools for broader ranges.

Conclusion

Kishuboard introduces a novel two-dimensional code+data version management framework, effectively addressing the lack of nonlinear exploration support in Jupyter Notebook. Through theoretical model analysis and user experiments, Kishuboard demonstrates significant improvements in productivity and user satisfaction, with promising potential for widespread application. However, future enhancements are needed to address collaboration features, semantic search, and large-scale comparison to further improve the tool’s practicality and user experience.