Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision Users

Document Title

Revamp: Enhancing Accessible Information Seeking Experience of Online Shopping for Blind or Low Vision Users

Document Information

• Subject Area: Accessible online shopping, information retrieval, human-computer interaction
• Keywords: online shopping, information retrieval, accessibility, visually impaired users, user reviews, image description, question-answering system

Research Background and Issues

• Problems or Challenges:

  • Visually impaired users face significant difficulties in online shopping, including insufficient image descriptions and the inability of screen readers to process large amounts of information effectively.
  • For product categories where appearance plays a crucial role in purchase decisions (e.g., fashion, home goods), existing automated tools provide overly generic information, making it hard to support decision-making.
  • Visually impaired users often rely on assistance from sighted individuals, such as family members or crowdsourced helpers, which is not always feasible and undermines shopping independence.

• Importance:

  • Online shopping is vital for visually impaired users, especially during the pandemic when demand has increased. Improving the accessibility of online shopping experiences is critical for enhancing quality of life.

• Research Motivation and Related Work:

  • Existing studies aim to support accessible online shopping through image descriptions and screen reader optimization but often fail to address the need for detailed visual information.
  • Automated information generation technologies (e.g., image recognition, visual question answering) still face limitations in providing personalized and detailed descriptions.
  • User-generated reviews may contain valuable information about product appearance, but how to systematically utilize this information remains unclear.

Solution

• Proposed Solution:

  • Develop an interactive information retrieval system named Revamp to support question-answering functionality based on customer reviews and restructure product pages.
  • Leverage the diverse content of user reviews to extract informative text segments using rule-based methods, generate image descriptions, answer user questions, and distinguish between positive and negative review content.

• Innovations:

  • Combine rule-based information retrieval with natural language processing, designing modular grammar rules tailored to the visual information needs of visually impaired users.
  • Provide a browser extension to simplify webpage structures, enhance screen reader functionality, and support independent online shopping.

• Implementation Steps and Key Technologies:

  1. Data Source: Scrape basic information such as titles, prices, and colors, along with review data, from Amazon.com pages.
  2. Rule Design:
     • Filter out low-content, less informative reviews (e.g., short sentences and comments on image consistency).
     • Extract sentences containing keywords related to visual attributes (color, logo, shape, size) and apply grammar rules to select descriptive and comparative text.
  3. Answer Generation: Provide a complete list of reviews categorized by positive and negative sentiment, summaries, and visual descriptions generated through ranking.
  4. Page Simplification and Reconstruction: Restructure webpages to retain only key product-related information.
  5. Interactive Design: Users can input questions via voice or text, and the system returns responses with summaries and a complete review list.

Research Outcomes

• Specific Results:

  • In experiments, the proposed rules covered 85% of informative review content and successfully provided useful visual attribute descriptions.
  • The Revamp prototype was deemed effective by visually impaired users in reducing the time and difficulty of information retrieval, improving online shopping independence.

• Advantages Over Existing Solutions:

  • Compared to Amazon's default review sorting mechanism, Revamp more efficiently extracts visually relevant information and provides targeted descriptions.
  • Offers two levels of detail (summaries and specific review lists) to better meet diverse information needs.

• Experiment and Evaluation Results:

  • Rule Evaluation: Across three product categories (Home & Kitchen, Clothing & Accessories, Electronics) and 45 best-selling items, the rule-based generation covered most visual-related questions.
  • System Evaluation: Eight participants rated Revamp significantly higher in efficiency and ability to understand product appearance.
  • Qualitative Feedback: Users appreciated the subjective details in reviews as a more trustworthy way to understand products and expressed interest in extending the system to other shopping platforms.

• Limitations and Future Directions:

  • Revamp relies on the quality of review data, and its effectiveness is limited when reviews are sparse or lack detail.
  • Future directions include:
    • Adding cross-product comparison functionality and enhanced multi-product information switching.
    • Expanding the method to other types of online content platforms (e.g., Yelp, TripAdvisor).
    • Collecting more annotated data to support supervised learning methods for advanced visual concepts and style inference.
    • Providing template-based questions to guide users in asking more precise queries.
    • Integrating into mobile and voice assistant applications to improve usability.

Conclusion

Revamp offers a solution to meet the visual information needs of online shopping through information retrieval technology, transforming user reviews into valuable visual descriptions via rule design and interactive systems. While its usability is still constrained by data quality, the system successfully improves the online shopping experience for visually impaired users, significantly enhancing information retrieval efficiency and autonomy. Future research can further expand functionality, optimize question-answering quality, and adapt to more platforms and user groups.