NBSearch: Semantic Search and Visual Exploration of Computational Notebooks

Document Title

NBSearch: Semantic Search and Visual Exploration of Computational Notebooks

Document Information

• Subject Area: Human-Computer Interaction and Information Retrieval, particularly code search in computational notebooks
• Keywords: Semantic code search, search result visualization, computational notebooks, documentation and text analysis, natural language processing, deep learning, code exploration, user interface design

Research Background and Problem

• Research Background: Computational notebooks (e.g., Jupyter Notebooks) are essential tools in modern programming activities, integrating code, documentation, and outputs into a single document to support dynamic development. However, due to their non-linear structure and high flexibility, managing and searching code in computational notebooks presents significant challenges.
• Main Problems:
  1. Traditional code search methods (e.g., keyword-based search) struggle to efficiently handle the loosely structured and non-linear code snippets in computational notebooks.
  2. The complex internal (within a notebook) and external (across notebooks) relationships among search results are difficult for existing search engines to clearly present.
• Research Significance: Effective code search is critical for developers to understand the context of code snippets, discover related code, and improve development efficiency.
• Motivation and Related Work:
  • Semantic code search allows developers to query code using natural language, but existing semantic search tools rarely focus on computational notebooks.
  • Previous tools have primarily addressed version control, history tracking, and other support functionalities, rather than efficient code search and visualization of search results.

Solution

• Core Approach:

  1. Introduced NBSearch, a new system supporting semantic code search and interactive visualization of search results.
  2. NBSearch employs deep learning techniques (e.g., seq2seq models) to generate natural language descriptors for code and uses the Doc2Vec language model to process natural language queries.
  3. Designed an innovative search result visualization interface (NBLines) to help developers understand complex relationships between code snippets.

• Innovations:

  • Implemented fine-grained, cell-level semantic search within computational notebooks.
  • Used NBLines to visually display relationships among code cells within and across notebooks, enabling interactive exploration.
  • Combined deep learning-based code translation and semantic embedding methods to enhance the semantic relevance of search results.

• Implementation Steps and Key Techniques:

  1. Building the Search Database:
     • Trained a seq2seq translation model using annotated code cells to "translate" code into corresponding textual descriptors.
     • For code cells lacking annotations, used the trained translation model to automatically generate descriptors.
     • Applied Doc2Vec to semantically represent all descriptors, creating a searchable database.
  2. Search and Result Presentation:
     • Upon receiving a natural language query, the system converts the query into an embedding vector using the Doc2Vec model and retrieves the most relevant descriptors.
     • Relevant code cells are visualized and interactively presented through NBLines.
  3. Iterative Design:
     • Conducted a six-month participatory design process with domain experts, collecting feedback through prototype testing to iteratively optimize the system.

Research Outcomes

• Specific Achievements:

  • Developed a fully functional NBSearch prototype, enabling developers to perform semantic code searches across large-scale notebook collections.
  • Proposed the NBLines visualization tool to display relationships among code snippets within and across notebooks.
  • Implemented a complete computational framework, from database preprocessing to front-end interface, optimizing the code search experience.

• Comparative Advantages over Existing Solutions:

  1. NBSearch supports natural language-based fuzzy search, addressing the limitations of existing code search tools.
  2. Tailored to the highly non-linear and flexible structure of notebooks, NBSearch refines the search granularity to the cell level and uncovers complex code relationships.
  3. The system enhances understanding and exploration of search results through interactive visualization, surpassing traditional list-based result displays.

• Experiments and Evaluation Results:

  1. Experimental Results: Evaluated the search engine's performance using multiple deep learning models (e.g., BLEU scores for the translation model).
  2. User Study: Conducted a user study with 12 experienced developers at Uber, yielding the following feedback:
     • NBSearch received positive evaluations for query efficiency, relevance of search results, and system usability.
     • The interactive features of NBLines helped developers quickly understand and explore code relationships.
  3. Performance Bottlenecks:
     • Some participants suggested that search results require more contextual information.
     • Certain user interface designs (e.g., dots view) posed comprehension challenges.

• Limitations and Future Directions:

  1. The current model only provides semantic descriptions for code cells and does not account for relationships with markdown annotations.
  2. Future work could expand support for sub-cell and multi-cell level searches.
  3. Users face a learning curve in understanding the system interface; improvements to NBLines' presentation and the addition of instant tutorials could address this.
  4. Training models on larger-scale datasets could enhance the precision and generalizability of retrieval results.

Conclusion

NBSearch demonstrates how deep learning and interactive visualization techniques can support semantic search and exploration of code relationships in computational notebooks. Future work could focus on improving model performance, enhancing interface design, and expanding application scenarios to provide developers with more efficient code management and search experiences.