TermSight: Making Service Contracts Approachable

Paper Title

TermSight: Making Service Contracts Approachable

Publication Info

• Topic area: Augmented reading interfaces for legal contracts
• Keywords: Terms of Service, legal contracts, augmented reading, language models, user interface design, comprehension, navigation, user study, legal text simplification, intelligent systems

Background and Problem

• Problem / challenge: Legal contracts, particularly Terms of Service (ToS), are difficult for non-experts to read due to their specialized, ambiguous, and legally binding language. Existing systems fail to address the unique challenges of navigating and understanding such documents.
• Significance: ToS govern critical aspects of digital interactions, yet most users do not read them due to their complexity. This lack of engagement undermines informed consent and personal autonomy in digital services.
• Motivation and related work: Prior work has focused on simplifying legal language or providing policy overviews but has not adequately addressed the challenges of navigating nested policies, resolving ambiguities, or maintaining the connection to legally binding text. This paper builds on these gaps by exploring multi-level guidance for ToS reading.

Solution

• Proposed approach: TermSight, an intelligent reading interface designed to make ToS contracts more approachable by providing multi-level guidance while preserving access to the original legally binding text.
• Novelty:
  1. Introduces a Power Meter to visualize the distribution of power and relevance across policies.
  2. Provides color-coded Summary Snippets for document-level navigation and understanding.
  3. Offers Phrase Scope for in-situ definitions and hypothetical scenarios to contextualize ambiguous terms.
  4. Maintains tight integration with the original text for verification and trust calibration.
• Procedure and key techniques:
  • Uses GPT-4 to generate concise summaries, classify power and relevance, and identify ambiguous phrases.
  • Implements a three-panel interface: a navigation panel, original text, and summaries.
  • Employs color coding to indicate power dynamics (red, yellow, green) and relevance (high/low saturation).
  • Allows users to navigate between summaries and original text with one click.

Results

• Concrete findings:
  • TermSight significantly improved user experience across seven measures, including ease of reading, navigation, and willingness to engage with ToS.
  • No significant differences in comprehension or recall compared to a baseline HTML reader.
  • Participants used features like Summary Snippets and Phrase Scope extensively, with Summary Snippets being the most frequently used.
• Advantage over baselines:
  • TermSight reduced navigation effort and made it easier to identify relevant information compared to a standard HTML reader.
  • Participants preferred TermSight (19/20) and reported that its features provided actionable guidance.
• Experiments / evaluation:
  • Conducted a within-subjects study with 20 participants comparing TermSight to a baseline interface.
  • Measured user experience, comprehension, recall, and feature usage.
  • Used Bayesian analysis to estimate treatment effects.
• Limitations and future work:
  • Did not improve comprehension or recall, possibly due to the overwhelming volume of information in ToS.
  • Study participants were mostly college-educated and fluent in English, limiting generalizability.
  • Future work could explore customization of user personas, adaptation to other contract types, and integration of validation mechanisms for AI outputs.

Summary

TermSight is an intelligent reading interface designed to make Terms of Service contracts more approachable by providing multi-level guidance, including visual overviews, concise summaries, and contextualized definitions. A user study demonstrated that TermSight significantly improved navigation and user experience, though it did not enhance comprehension or recall. The system's features, such as the Power Meter and Summary Snippets, were well-received and frequently used. While TermSight highlights the potential of AI-powered tools for legal text, challenges like information overload and the need for policy innovation remain. Future work could expand its applicability to other contract types and user populations.