Designing and Evaluating Interfaces that Highlight News Coverage Diversity Using Discord Questions

Title of the Paper

Designing and Evaluating Interfaces to Highlight News Diversity: Exploring Discord Questions

Bibliographic Information

• Subject Area: Human-Computer Interaction and News Information Visualization
• Keywords: News reading interface, report diversity, discord questions, natural language processing, human-computer interaction, usability study, news bias, multi-perspective, user experience, news data visualization

Research Background and Issues

• Core Issues and Motivation:

  • Modern news aggregators can compile a large number of reports on the same event from multiple sources, but users need to spend significant time and effort filtering and comparing information from these sources.
  • In such multi-source environments, a lack of transparency may expose readers to biased information, particularly on major societal issues such as elections or international affairs.
  • Current news reading interfaces often treat news articles as atomic units, lacking targeted support for revealing and comparing differences in reporting.
  • This paper proposes designing new reading interfaces to reveal the diversity of reporting, thereby helping readers discover richer information in news content.

• Research Significance:

  • Having diverse news perspectives is a key prerequisite for ensuring information transparency and a deep understanding of complex issues.
  • By integrating natural language processing (NLP) techniques to automatically extract differences in news reporting, the effort required by readers to compare multi-source information can be reduced.

Solution

• Proposed Methods:

  • This paper designs three types of news reading interfaces:
    1. Annotated Article: Adds annotations based on NLP-extracted discord questions and multi-source answers to existing single news articles.
    2. Recomposed Article: Automatically synthesizes a question-and-answer format article from multiple news reports, covering more information.
    3. Question Grid: Displays a matrix of questions (rows) and corresponding answers from sources (columns), emphasizing overall information density.
  • Utilizes the "Discord Questions Framework" to automatically generate diverse questions and their answers, revealing differences and potential conflicts between reports.

• Innovative Contributions:

  • Introduces NLP-generated "discord questions" as the core unit, combined with a question-driven reading approach to showcase differing perspectives on the same issue from various sources.
  • The three interfaces reflect increasing complexity: from simple annotations on existing news to high-density information visualization, catering to the needs of both novice and expert users.
  • The proposed interfaces systematically test the practical effectiveness of automatically extracted discord questions in news reading for the first time.

• Key Technologies Used:

  • NLP techniques, including Question Generation, Answer Extraction, and Answer Consolidation, implemented using pre-trained language models such as BART and RoBERTa.
  • Usability Study, employing human-computer interaction experiments to evaluate and optimize interface design.

Research Findings

• Main Results:

  • The proposed Annotated Article interface achieved the best balance between readability and diversity presentation, enabling users to more clearly grasp diverse perspectives in reporting.
  • Compared to unmodified single news articles or simple headline list interfaces, users of the Annotated Article interface demonstrated a 34% improvement in the completeness of their answers in reading comprehension tests.
  • The more complex Recomposed Article and Question Grid interfaces exhibited higher information density but posed certain usability challenges for novice users.

• Experimental Evaluation:

  • Expert Evaluation:
    • A study involving 10 news professionals indicated that the Annotated Article interface is suitable for most users, while the Question Grid is more appropriate for professional news analysis.
  • User Testing:
    • In tests involving 95 ordinary news readers, the Annotated Article interface significantly improved the comprehensiveness of news understanding while maintaining ease of use.

• Limitations and Future Directions:

  • Automatically generated discord questions occasionally contained noise or incorrect answers, limiting the method's practical application.
  • Most experiments were conducted on English news data, presenting language and regional limitations; future work should expand to multilingual environments.
  • The contextual presentation of news can be further optimized, such as improving semantic and topical coherence in Recomposed Articles.
  • Future longitudinal studies could explore the impact of these interfaces on users' long-term news consumption habits.

Output Format

• Publication Platform: CHI 2023 (Top-tier conference in the field of Human-Computer Interaction).
• Experimental Dependencies: NLP-based algorithms, two-stage usability studies (including expert and general user groups).
• Social Value: Enhances transparency in news reading, ensures comprehensive and in-depth understanding of complex events for news users, and promotes counteraction against news bias and the spread of polarized information.