InSupport: Proxy Interface for Enabling Efficient Non-Visual Interaction with Web Data Records

Title of the Paper

InSupport: Proxy Interface for Enabling Efficient Non-Visual Interaction with Web Data Records

Bibliographic Information

• Subject Area: Accessibility Technology, Non-Visual Interaction with Web Data Records
• Keywords: Web Accessibility, Blind Users, Visual Impairment, Screen Readers, Data Records

Research Background and Problem

• Identified Problems:

  • Blind users experience inefficiency when interacting with web data records (e.g., products, flight information, posts) via screen readers, particularly when navigating auxiliary segments (e.g., filters, sorting options, search forms). Frequent switching is required, making operations cumbersome.
  • Screen reader navigation is based on a one-dimensional model, causing users to frequently toggle between auxiliary segments and data records, increasing interaction burden.
  • Current solutions focus primarily on general navigation of web content, with limited support for specific operations on data records (e.g., filtering and sorting).

• Significance:

  • Accessible web browsing is crucial to ensuring equal access to digital content for visually impaired individuals, but accessibility alone cannot meet the usability needs of blind users.
  • Web interfaces are primarily designed for sighted users, often requiring blind users to spend more time and effort to perform the same tasks.

• Research Motivation and Related Work:

  • Existing methods (e.g., voice interaction and web automation technologies) have limitations, such as sensitivity to environmental noise, privacy concerns, and the additional burden of maintaining task scripts.
  • Some studies propose using artificial intelligence to annotate "hotspot" areas on web pages, but these methods are ineffective in significantly improving the efficiency of operations specific to data records.

Solution

• Proposed Method:

  • Designed and implemented a browser extension named InSupport, which:
    1. Uses a custom machine learning algorithm to automatically extract auxiliary segments from web pages containing data records.
    2. Provides a touch-enabled proxy interface, enabling quick navigation of auxiliary segments using simple screen reader shortcuts.

• Innovations:

  • Introduced a machine learning-based detection algorithm capable of efficiently identifying and extracting auxiliary segments (filters, sorting options, search forms, multi-page links) from the web page DOM tree.
  • Developed a non-visual proxy interface that simplifies switching between auxiliary segments and data records for blind users.
  • The user interface design supports beginners and non-tech-savvy users, requiring only a few basic shortcuts for efficient use.

• Implementation Steps and Key Technologies:

  1. Auxiliary Segment Extractor:
     • Utilized DOM analysis and machine learning algorithms (e.g., Logistic Regression and Multi-Layer Perceptron) to identify auxiliary segments.
     • The algorithm relies on handcrafted features, such as checking DOM attribute values for specific keywords.
     • Trained the model using a dataset of 209 manually annotated web pages.
  2. Proxy Interface:
     • Users activate the interface using a shortcut (e.g., CTRL+SHIFT+Z) and navigate auxiliary segments and options using arrow keys or the TAB key.
     • Once a filter or sorting rule is selected, the system automatically applies the corresponding action to the actual web page.
     • The interface design avoids unnecessary web exploration and optionally supports direct navigation via screen readers.

Research Outcomes

• Specific Results:

  • InSupport successfully implemented automatic extraction of auxiliary segments (filters, sorting options, search forms, multi-page links) and presented an adapted non-visual interaction interface in the browser.
  • User task analysis showed a significant reduction in operation time and steps compared to existing solutions.

• Comparative Advantages over Existing Solutions:

  • In user studies, InSupport demonstrated significantly improved efficiency compared to standalone screen readers and a state-of-the-art visual saliency-based model, SaIL:
    • Average task completion time reduced by 47%.
    • Average number of keystrokes reduced by 54%.
  • System usability (SUS score of 86.07) and perceived workload (TLX score of 25.61) outperformed comparison methods.

• Experiment or Evaluation Results:

  • User studies involved 14 blind participants performing two tasks (searching travel records based on predefined criteria and shopping records based on user preferences).
  • Results showed that InSupport significantly outperformed traditional screen readers and SaIL in terms of task time, keystrokes, system usability (SUS score), and workload (NASA-TLX metrics).

• Limitations and Future Directions:

  • Limitations:
    • Tested only on English web pages; the effectiveness of specially designed features in multilingual environments is not guaranteed.
    • Currently limited to the desktop Chrome browser, with no support for other browsers or mobile devices.
    • Performance on web pages where the extraction algorithm fails has not been tested.
  • Future Directions:
    • Enhance the algorithm to adapt to multilingual environments.
    • Develop cross-browser and mobile interface versions.
    • Introduce intelligent filtering strategies to recommend filters based on user history.
    • Explore preloading multi-page data records to reduce page-flipping operations.

Conclusion

InSupport provides a novel approach to reducing the operational burden of blind users when interacting with web data records. By leveraging a proxy interface, it significantly improves task efficiency and convenience. Future research can further expand automation and cross-platform functionality, continuously lowering the technical barriers to accessible interaction.