FreeForm: Flexibly Augmenting Formulas with Synchronized Markup and Graphical Edits

Research Background and Problem

• What problems or challenges did the authors identify?
  The expression of mathematical formulas is a crucial part of scientific communication, but understanding them often becomes difficult due to their complexity or unfamiliar symbols. Existing tools (such as LaTeX and graphic editors) have limitations in enhancing and annotating mathematical formulas: LaTeX offers precise syntax but is cumbersome to operate, while graphic editors are more intuitive but cannot synchronize with the formula structure. Some users have pointed out that debugging nested brackets, editing annotations, and realigning annotations after formula changes are particularly time-consuming tasks.

• Why is this problem important?
  Enhancing mathematical formulas (e.g., color annotations, labels, spacing adjustments) is critical for the readability of scientific documents, especially in educational and research contexts. Addressing this issue can significantly improve formula readability and reduce the workload for authors.

• Research Motivation and Related Work
  Previous research has proposed tools such as the FFL language and structured editing tools to optimize formula enhancement in LaTeX. While these tools simplify certain enhancement tasks, they have high learning costs or lack support for direct interaction. Additionally, projectional editing tools with multi-view synchronization have proven effective in other domains but have not been specifically designed for mathematical formulas. The authors aim to integrate the expressive power of LaTeX with the intuitiveness of graphical editing through FreeForm to address these issues.

Solution

• What methods or solutions did the authors propose?
  The authors developed FreeForm, a projectional editor-based tool that supports formula enhancement and editing through multiple synchronized representations (LaTeX code, rendered formulas, and tree structure views). Users can enhance formulas through graphical interactions, and these enhancements are automatically synchronized with the LaTeX code, allowing direct export or manual editing.

• What are the innovative aspects of this solution?

  1. Synchronization Mechanism: FreeForm achieves real-time bidirectional synchronization between graphical editing and LaTeX code, ensuring that enhancements are accurately mapped in both representations.
  2. Projectional View Design: It provides three synchronized views—code, rendered formulas, and tree structure—allowing users to freely switch between graphical and code-based editing as needed.
  3. Tree-Aware Selection: Supports tree-structured selection, enabling users to select subsets of complex formulas in a structured manner and apply enhancements.
  4. Diverse Enhancement Options: Offers various enhancement options, including color, labels, spacing alignment, and borders, with intuitive menu-based interactions.

• What are the implementation steps and key technologies used?
  The authors implemented FreeForm's front-end interface using React.js and employed the MathJax library for rendering LaTeX formulas. Additionally, the system uses an abstract syntax tree (AST) to synchronize and transform formula structure annotations and enhancements, while the CodeMirror library supports real-time LaTeX editing.

Research Outcomes

• What specific outcomes were achieved?

  1. FreeForm significantly accelerated the completion time for color and label enhancement tasks, showing clear advantages over two baseline tools (Overleaf and FFL).
  2. Observational studies revealed that participants extensively utilized FreeForm's multi-view functionality to quickly switch between editing workflows.
  3. Users expressed high satisfaction with the real-time preview of graphical editing and LaTeX code, particularly in reducing debugging time for complex annotation tasks.

• What advantages does it have compared to existing solutions?

  1. Compared to traditional LaTeX editors, FreeForm is more user-friendly, especially for annotating complex formulas without manually handling nested brackets.
  2. While FFL is better suited for applying complex enhancement rules across documents, FreeForm excels in direct interaction and localized enhancement tasks, significantly speeding up operations.
  3. Graphical editing reduces the cognitive load of remembering enhancement macro names while maintaining synchronization between annotations and formulas.

• What were the experimental or evaluation results?
  In comparative experiments, FreeForm was shown to significantly reduce the completion time for color and label enhancement tasks (by approximately 1.2 to 3 minutes). Participants also reported lower task difficulty. Observational studies further revealed users' actual workflows, including rapid enhancement, exploratory enhancement, and precise formula modifications using tree-structured selection.

• Limitations and Future Directions

  1. Limitations:
     • FreeForm's LaTeX synchronization mechanism may lead to loss of work content when users write temporary or incomplete code, causing unnecessary frustration.
     • The tool does not support automatic propagation of enhancement rules across documents (e.g., consistent symbol coloring), limiting its application scope.
     • Currently, users need to switch between FreeForm and other document editors (e.g., Overleaf), increasing operational overhead.
  2. Future Directions:
     • Integrate FreeForm with comprehensive document editors, allowing users to trigger enhancement operations directly within their workflows.
     • Expand enhancement options, such as supporting more font styles, background colors, and more flexible label annotations.
     • Improve the synchronization mechanism to allow editing under invalid LaTeX syntax without losing content.
     • Add cross-document enhancement propagation capabilities, such as automatically applying consistent enhancement rules across an entire document.

By providing actionable suggestions and expansion plans, FreeForm offers significant insights for future mathematical formula enhancement tools and related human-computer interaction research.