What We Talk About When We Talk About Frameworks in HCI

Paper Title

What We Talk About When We Talk About Frameworks in HCI

Publication Info

• Topic area: The role, usage, and lifecycle of frameworks in Human-Computer Interaction (HCI).
• Keywords: HCI frameworks, theoretical contributions, systematic review, framework lifecycle, framework validation, framework reuse, framework creation, CHI proceedings, meta-research, design frameworks.

Background and Problem

• Problem / challenge: Despite the prevalence of frameworks in HCI, there is limited understanding of how they are used, constructed, validated, and articulated for reuse. The field lacks systematic validation and cumulative practices for frameworks.
• Significance: Frameworks are essential for organizing knowledge, guiding design, and fostering interdisciplinary collaboration in HCI. Understanding their lifecycle and improving their development practices can enhance the field’s methodological rigor and practical impact.
• Motivation and related work: Prior studies have analyzed individual frameworks or classified their functions, but a comprehensive, data-driven typology of frameworks in HCI is missing. This paper builds on earlier meta-research and literature syntheses to address this gap.

Solution

• Proposed approach: A systematic review of 615 CHI papers (2015–2024) to classify framework engagement, analyze their characteristics, and evaluate their lifecycle practices.
• Novelty:
  1. Identifies six primary types of engagement with frameworks: Create, Adapt, Validate, Review, Use, and Mention.
  2. Proposes a functional typology of frameworks based on their roles: Descriptive, Explanatory, Predictive, Prescriptive, and Generative.
  3. Highlights gaps in validation and articulation practices for frameworks in HCI.
  4. Provides actionable recommendations for proposing, validating, and reusing frameworks.
• Procedure and key techniques:
  • Conducted a systematic review of CHI proceedings (2015–2024) using PRISMA guidelines.
  • Developed a codebook to classify engagement types and contribution types.
  • Used inductive and deductive coding, keyword embedding, and clustering to analyze framework characteristics and thematic domains.
  • Synthesized findings into a functional typology and analyzed methodological practices for framework construction, validation, and articulation.

Results

• Concrete findings:
  • 54.6% of papers create new frameworks, while only 7.15% adapt, 3.9% validate, and 0.65% review existing frameworks.
  • Frameworks are often poorly validated, with many lacking explicit reuse instructions or success criteria.
  • Functional roles of frameworks include organizing phenomena (Descriptive), explaining causal relationships (Explanatory), forecasting outcomes (Predictive), providing actionable guidance (Prescriptive), and inspiring new ideas (Generative).
• Advantage over baselines:
  • Provides the first large-scale, data-driven typology of framework engagement in HCI.
  • Identifies critical gaps in iterative refinement, validation, and reuse practices.
• Experiments / evaluation:
  • Analyzed 615 CHI papers using a structured coding protocol and thematic clustering.
  • Validated coding reliability with inter-rater agreement (Cohen’s κ ≈ 0.78 for engagement types).
  • Mapped framework contributions across nine thematic clusters in HCI.
• Limitations and future work:
  • Focused on CHI proceedings; other HCI venues may exhibit different practices.
  • Relied on authors’ explicit use of the term “framework,” potentially missing implicit contributions.
  • Future work could include citation analysis and deeper studies of specific framework types or subfields.

Summary

This paper systematically reviews 615 CHI papers (2015–2024) to analyze the role and lifecycle of frameworks in HCI. It identifies six types of engagement (Create, Adapt, Validate, Review, Use, Mention) and proposes a functional typology based on five roles (Descriptive, Explanatory, Predictive, Prescriptive, Generative). The study reveals a dominance of framework creation over validation and reuse, highlighting gaps in systematic validation and articulation for reuse. Recommendations are provided to improve framework development and adoption practices, fostering a more cumulative and rigorous research culture in HCI.