Using Annotations for Sensemaking About Code

Document Title

Using Annotations for Sensemaking About Code

Document Information

• Subject Area: Annotation tools in software development and code comprehension
• Keywords: annotation tools, lightweight notes, software engineering, code comprehension, sensemaking, debugging, developer workflow, information externalization

Research Background and Issues

• Identified Problems or Challenges:

  1. Developers need to manage large amounts of information when understanding and debugging code, including task objectives, points of interest in the code, issues with code behavior, and hypotheses;
  2. Programming tasks may span multiple days and require developers to maintain task context, facing difficulties in resuming work after interruptions;
  3. Existing tools (e.g., code comments and external notes) are insufficient for supporting information externalization and tracking, prone to loss, lacking contextual connections, and potentially causing code clutter.

• Importance: Efficient information management is critical for the success of software development, enabling developers to understand code functionality faster, reduce cognitive load, and collaborate effectively on tasks.

• Research Motivation and Related Work:

  • Code and task annotations are commonly used to address these issues but have the following limitations:
    • Annotations may need to be cleaned up to avoid clutter;
    • Certain scenarios prevent the use of annotations (e.g., external library code or restrictions on modifying source code).
  • Developers tend to use external tools and notes, but these methods are often not sufficiently linked to the code, leading to information gaps.

Solution

• Proposed Method or Solution: Develop and introduce a lightweight code annotation tool—"Catseye." This tool allows developers to externalize information within the context of the code without altering the source code.

• Innovative Features:

  1. Annotation functionality surpassing traditional code comments:
  • Supports unified annotation syntax across languages;
  • Can be bound to any text or code range;
  • Enables threaded conversations to track developer feedback and answers.
  1. Additional features:
  • Lightweight version management;
  • System output association and search functionality;
  • Temporarily "pinned" annotations for navigation across code segments.

• Implementation Steps and Key Technologies:

  1. Annotation Creation: Use shortcuts, menus, or command bars to select code segments and create annotations;
  2. Annotation Features:
     • Multi-location anchor creation;
     • Reply system;
     • Quick navigation;
     • Code snapshots;
     • Search functionality;
  3. Technical Implementation:
     • Developed using TypeScript and React;
     • Integrated with Visual Studio Code API;
     • Utilized Google Firestore to store annotation data.

Research Results

• Specific Outcomes:

  1. Introduced the Catseye tool and embedded it into the Visual Studio Code editor;
  2. Demonstrated through experiments that Catseye effectively aids developers in debugging tasks, facilitating easier information externalization.

• Advantages Over Existing Solutions:

  • Annotations do not modify the source code and do not affect code execution;
  • Flexibly adapt to developers' personalized information needs;
  • Provide convenient navigation and task tracking features;
  • Integrate context-related information to enhance efficiency.

• Experiment or Evaluation Results:

  • Participants using Catseye achieved significantly higher success rates in debugging tasks compared to the control group;
  • The average frequency of annotation interactions was approximately twice that of the control group, indicating that annotations promoted information revisitation and usage;
  • Participants experienced a significant reduction in cognitive load during tasks and were better able to mark and track information.

• Limitations and Future Directions:

  • Limitations:
    • Experimental tasks were based on artificially designed codebases, not real-world scenarios;
    • Does not support graphical outputs or more complex interaction methods.
  • Future Directions:
    • Enhance integration with GitHub to support collaborative development;
    • Optimize annotation and code version update management;
    • Expand support for capturing other media formats, such as adding charts or screenshots.

This study is the first to demonstrate how code annotation tools can serve as a cognitive workspace for developers, acting as essential tools for debugging, sensemaking, and information externalization. The research provides profound insights into tool design and developers' information needs, offering significant practical and academic value.