Journalistic Source Discovery: Supporting The Identification of News Sources in User Generated Content

Title of the Paper

Journalistic Source Discovery: Supporting The Identification of News Sources in User Generated Content

Paper Information

• Subject Area: Discovery and support of news sources in User Generated Content (UGC)
• Keywords: Computational journalism, Crowdsourced journalism, Citizen journalism, User Generated Content, News source filtering, Information perception

Research Background and Issues

• Identified Problems and Challenges:

  • The traditional news industry increasingly relies on User Generated Content (UGC) as news material, but efficient and accurate discovery of news sources from UGC has not been deeply studied or effectively supported by tools.
  • Journalists face challenges in filtering information, assessing relevance, and evaluating credibility amidst the vast amount of UGC.
  • Many UGC items may not align with the conventions and values of traditional newsrooms, posing difficulties in integrating tools for effective practice.

• Significance:

  • UGC is full of potential, offering unique perspectives, insights, and expert opinions. It can complement traditional news reporting, extract valuable information, and promote deeper and more diverse journalism.
  • At the same time, challenges lie in managing the scale and noise of UGC and preventing the spread of misleading information.

• Research Motivation and Related Work:

  • Existing research primarily focuses on monitoring, verification, and trend detection in UGC, but support for discovering news sources is relatively lacking.
  • By designing innovative tools, bridging the technological gap between journalists and UGC can significantly improve the efficiency of UGC utilization and expand its potential.

Solution

• Methods and Solutions:

  • Conducted qualitative interview studies with 9 professional journalists to analyze UGC source needs and practices, summarizing two methods for discovering news sources:
    • Deep Reporting.
    • Wide Reporting.
  • Developed a prototype tool based on the study results to help journalists interactively filter and rank UGC based on specific content.
  • Provided diverse design goals, including representative ranking, interactive re-ranking, and grouping with diversity.

• Innovative Aspects of the Solution:

  • Defined and implemented a computational model of "representativeness," which calculates the average semantic centrality distance of UGC posts to capture the average degree of UGC content.
  • Introduced a dynamic interactive hierarchical algorithm based on semantic similarity, allowing users to refine queries through examples or selections.
  • Cluster analysis assists journalists in exploring community discussions from different perspectives, providing more efficient data organization and understanding support.

• Implementation Steps and Key Technologies:

  • Used BERT word embeddings to construct document semantic representations.
  • Designed and implemented a hierarchical ranking algorithm (dynamic re-ranking via cosine similarity).
  • Applied K-means for cluster analysis and used the TF-IDF method to automatically generate keyword labels for each cluster.
  • The prototype interface provides core functionalities such as data loading, group filtering, query examples, and ranking refresh.

Research Outcomes

• Specific Outcomes:

  • Identified journalists' basic needs for discovering news sources in UGC: personal experience and expertise, community responses and trends, issues, and different perspectives and opinions.
  • Proposed operational definitions for Deep Reporting and Wide Reporting, summarizing their characteristics and use cases.
  • Created a prototype tool to help media professionals mine data from UGC, significantly improving news discovery efficiency through semantic analysis.

• Advantages Compared to Existing Solutions:

  • Compared to traditional keyword-based search methods, the prototype tool extracts more relevant results based on deep semantics.
  • Offers representativeness ranking and interactive re-ranking functionalities, effectively addressing the scalability challenges of UGC.
  • Supports the creation of group-based insights and multi-perspective interpretations, which are lacking in traditional UGC analysis tools.

• Experimental or Evaluation Results:

  • In interactive evaluations with 9 journalists, users generally found the tool outstanding in saving time, discovering new angles, and supporting community response analysis.
  • Journalists considered "representativeness ranking" and "semantic similarity re-ranking via input/selection" particularly useful, greatly aiding actual interviewing and writing tasks.
  • Suggestions for improvement included enhancing the clarity of cluster labels, supporting multi-group selection, and adding transparency and algorithm customization features.

• Limitations and Future Directions:

  • Limitations:

    • The study was limited to one user scenario (Wide Reporting), with Deep Reporting not extensively investigated.
    • Test data was sourced from a single platform (The New York Times), leaving applicability to other UGC platforms unverified.
    • The tool's algorithm did not adjust for different editorial values, falling short of fully realizing personalized news preferences.

  • Future Directions:

    • Develop tools and a comprehensive technical framework to support different types of news content (e.g., investigative reporting).
    • Integrate more information sources and metadata (e.g., geographic, temporal, sentiment tags) to optimize user interaction data.
    • Explore automated enhancement technologies for text generation and keyword annotation.
    • Expand the research scope to include long-term, real-world usage scenarios for more ecological analysis and optimization.