Relatedly: Scaffolding Literature Reviews with Existing Related Work Sections

Title of the Paper

Relatedly: Scaffolding Literature Reviews with Existing Related Work Sections

Paper Information

• Subject Area: Literature Reviews, Scientific Discovery, Exploratory Search, Human-Computer Interaction
• Keywords: Literature Review, Scientific Discovery, Exploratory Search, Sensemaking, Human-Centered Computing

Research Background and Problem

• Challenges Identified:

  • With the exponential growth of scientific literature, researchers face cognitive overload and information overload when reviewing large volumes of literature to understand a topic.
  • Current literature review tools (e.g., academic search engines and visualization tools) provide limited support and fail to effectively help researchers understand connections between different papers or gain an overview of a topic.
  • The "related work" sections in academic papers offer good summaries of literature but are typically limited to supporting a single paper and do not comprehensively cover a specific domain.

• Significance of the Problem:

  • Literature reviews are a critical part of scientific research, essential for identifying research patterns and gaps, establishing the rationale for new studies, and defining research objectives.
  • Addressing these challenges can not only enhance researchers' productivity but also accelerate scientific discovery.

• Motivation and Related Work:

  • Existing tools (e.g., Google Scholar) primarily focus on locating individual papers but lack functionalities to synthesize content across multiple papers.
  • Automatically generated clusters or summary tools often suffer from inaccuracies and unclear themes.
  • Information foraging theory provides insights into optimizing information gain, and Relatedly aims to address literature review challenges by prioritizing unexplored and diverse information through reordering and highlighting.

Solution

• Method or Solution:

  • Leverage the "related work" sections of papers to provide users with structured exploration of scientific topics, including features such as dynamic reordering, highlighting unexplored and redundant information, and automatically generating paragraph titles.
  • Design the Relatedly system to organize "related work paragraphs," enabling users to review topics systematically and create structured review outlines.

• Innovations:

  • Unlike existing academic search engines, this system extracts information from the "related work" sections of existing literature, reducing the workload of starting a review from scratch.
  • Relatedly employs a dynamic reordering algorithm to maintain query relevance while highlighting unexplored, diverse information, offering users a more comprehensive perspective.
  • By combining automatic title generation and reading support features, the system addresses issues of redundant information and difficulty in tracking exploration progress.

• Implementation Steps and Key Technologies:

  • Automatic Paragraph Title Generation: Based on the BART model, generate descriptive titles for paragraphs without existing titles.
  • Paragraph Reordering: Use BM25 and Maximal Marginal Relevance (MMR) techniques to ensure that the initial paragraphs users browse contain the most diverse and unexplored references.
  • Information Marking and Progress Tracking: Highlight unexplored references, gray out already-read content, and display a progress bar for exploration tracking.

Research Outcomes

• Specific Results:

  • Experiments show that users of the Relatedly system can generate more structured, in-depth, and comprehensive topic outlines.
  • Compared to baseline conditions, Relatedly users accessed over twice as much paragraph content and gained a better understanding of connections between papers and different themes.
  • Users reported that reading related work paragraphs helped them design higher-quality review structures, providing semantic understanding and efficient exploration.

• Advantages:

  • Compared to traditional literature exploration methods, such as reading individual papers or abstracts, Relatedly significantly improves exploration efficiency.
  • Users rated their overall experience with Relatedly significantly higher, including its support for discovering related research, understanding relationships between concepts, synthesizing information from multiple sources, and tracking reading progress.

• Experimental or Evaluation Results:

  • In a comparative test with 15 participants, the quality of review outlines generated by Relatedly users received higher expert evaluations, including better coherence, insightfulness, and detailed descriptions of topics.
  • Users interacted more frequently with paragraph content and references, demonstrating their utilization and acceptance of Relatedly's features.

• Limitations and Future Directions:

  • The current work relies on high-quality related work sections, and further development may be needed to assess the quality of literature in cases of poorly written or narrowly scoped papers.
  • Future work could support exploration across multiple topics and queries, extending to the needs of non-academic users, such as policymakers or non-experts browsing scientific information.
  • There is potential to design more intelligent recommendation mechanisms or optimize algorithms to meet users' personalized information needs.