The Sharply Decreasing Disruptiveness of HCI

Research Background and Issues

• Identified Problems or Challenges: Although creativity has long been a significant theme in Human-Computer Interaction (HCI) research, the inherent creativity of HCI itself and its evolution remain underexplored. Over time, the creativity within the research domain appears to be declining.
• Importance of the Issue: Creativity in the research domain is crucial for individual researchers to identify promising directions and position their contributions, and it is also vital for the healthy development of the entire HCI community.
• Research Motivation and Related Work:
  • Traditional creativity research has primarily focused on developing specific tools to enhance practitioners' creativity, lacking critical examination of the inherent creativity within HCI research.
  • The recently introduced "Disruptive Index" provides a novel evaluation dimension, quantifying the impact of innovation on existing literature through changes in citation networks, thereby assessing its creativity.

Solution

• Proposed Solution:
  • Conduct a large-scale bibliometric analysis of citation networks from major HCI conference papers between 1982 and 2023 to quantify their disruptiveness and explore underlying contributing factors.
  • Utilize the Disruptive Index as the evaluation method, analyzing topics, knowledge usage, and author backgrounds to investigate the relationship between disruptiveness and creativity.
• Innovative Aspects:
  • Introduce quantitative bibliometric analysis and multidimensional disruptiveness measurement methods to reveal the evolution of disruptiveness in HCI research over time.
  • Combine author "freshness" and the knowledge base of papers to provide new perspectives on the issue of declining creativity in HCI.
• Implementation Steps:
  1. Collect metadata of HCI papers (sourced from ACM and OpenAlex).
  2. Calculate the Disruptive Index for each paper.
  3. Perform topic analysis using objective topics (OpenAlex) and subjective keywords.
  4. Analyze citation patterns of papers (e.g., citation counts, age, and popularity of cited references).
  5. Evaluate the relationship between author background characteristics and disruptiveness (e.g., career age and freshness of authors).

Research Findings

• Specific Findings:
  • Significant decline in disruptiveness within HCI: The proportion of disruptive papers dropped from over 80% in 1982 to around 20% in 2023.
  • Citation patterns of disruptive papers: Disruptive research trends tend to rely on fewer, older, and less popular references.
  • Disruptiveness and team structure: Teams with more new authors and shorter career ages are more likely to produce disruptive papers.
• Comparison with Existing Solutions:
  • Compared to other disciplines, the decline in disruptiveness within HCI is notably faster, especially in top-tier conference papers.
• Experimental or Evaluation Results:
  • Disruptive papers are more likely to achieve exceptionally high citations (e.g., entering the top 1% of citation counts), though their average citation counts may not necessarily be higher.
  • The primary contributors of disruptive papers remain the United States, but emerging players like China are beginning to stand out.
• Limitations and Future Directions:
  • Unpublished papers and the impact of the review process on disruptiveness were not considered.
  • The study did not specifically address regional particularities and their contributions to disruptiveness.
  • Future research could explore other dimensions of creativity and delve deeper into how review mechanisms support or hinder innovation.

Conclusion and Recommendations

• Key Conclusions: The disruptiveness in the HCI field is undergoing a significant decline, particularly in top-tier conferences, while research with fresh perspectives and greater diversity clearly has the potential to enhance disruptiveness.
• Recommendations:
  • Encourage the academic community to embrace new methodologies and unconventional research directions.
  • Support high-risk, innovative research and strengthen guidance and support mechanisms for new researchers.
  • Improve global participation (especially for researchers from developing countries) to foster diversity and creativity in the HCI field.
  • Increase recognition of representative works rather than solely focusing on quantity or average citations.

These findings and analyses provide valuable insights and practical directions for the HCI community to enhance creativity in the future.