A Critique of Electrodermal Activity Practices at CHI

Title of the Paper

A Critique of Electrodermal Activity Practices at CHI

Paper Information

• Field of Study: Human-Computer Interaction (HCI), methodological evaluation with a focus on Electrodermal Activity (EDA).
• Keywords: Electrodermal Activity, Galvanic Skin Response, EDA, GSR, Human-Computer Interaction, Experimental Design, Data Analysis, Methodology, Data Transparency, Signal Processing.

Research Background and Issues

• Issues or Challenges:

  1. Electrodermal Activity (EDA) is widely used in HCI research as a tool to measure psychological states such as arousal, attention, and stress.
  2. The authors identified significant methodological issues in how the HCI community handles EDA data, including experimental design, data collection and analysis, and reporting transparency.
  3. While the psychophysiology field has established mature standards for EDA analysis, the adoption of these standards in HCI remains insufficient.

• Significance:

  • If EDA usage does not adhere to standardized methods, the validity of research findings may be compromised, affecting the academic contributions based on EDA.
  • Enhancing methodological transparency and consistency is crucial for ensuring research reproducibility and external validity.

• Research Motivation and Related Work:

  • The study is motivated by the potential of EDA in HCI and the limitations in its methodological application. There is a growing demand within the community for transparent data analysis and experimental reproducibility.
  • Previous psychophysiology studies have detailed standards for EDA analysis, but the quality of application and reporting in the HCI community has not been thoroughly evaluated.

Solution

• Methods/Solutions:

  • The authors conducted a systematic literature review of 41 papers involving EDA data published in the ACM CHI proceedings between 1986 and 2020.
  • Using a coding scheme based on psychophysiology standards, they assessed research quality in areas such as equipment, preprocessing and controls, participant information, terminology usage, data analysis, and psychological constructs.

• Innovations:

  • Developed a detailed checklist of issues and standards based on the reviewed literature.
  • Proposed specific recommendations to improve EDA methodologies in the HCI community by integrating psychophysiology standards.

• Implementation Steps:

  1. Filtered papers containing EDA-related keywords from the ACM CHI database.
  2. Categorized and coded the papers based on publication standards recommended by psychophysiology, evaluating practices in equipment usage, experimental design, and data analysis.
  3. Summarized prevalent issues and proposed community-driven guidelines.

Research Outcomes

• Specific Findings:

  1. Identified major issues across the 41 papers, including missing information on equipment selection, insufficient experimental controls, and deviations from standard terminology and analysis methods.
  2. Developed a checklist of 40 issues covering equipment details, participant data, external factor controls, and signal processing.
  3. Created a set of community-curated guidelines (see https://edaguidelines.github.io/) to enhance experimental transparency and methodological standardization.

• Advantages Compared to Existing Solutions:

  • Transferred mature standards from psychophysiology to HCI, addressing methodological gaps in EDA usage within HCI.
  • Provided comparative analyses showcasing best practice examples and problematic cases.

• Experimental or Evaluation Results:

  • Among the 41 papers, only 3 fully adhered to psychophysiology-recommended terminology standards.
  • Over half of the studies lacked detailed descriptions of the equipment and signal processing methods used.
  • Only 12 papers met standards for data preprocessing and signal parameterization.

• Limitations and Future Directions:

  • Limitations:
    1. The sample is limited to CHI proceedings, lacking broader representation of the HCI field.
    2. Psychophysiology standards may be overly stringent and not fully aligned with HCI's diverse objectives.
  • Future Directions:
    1. Develop new wearable EDA devices suitable for everyday scenarios.
    2. Validate existing devices and improve their accuracy.
    3. Create more user-friendly real-time signal processing tools to reduce learning barriers.
    4. Promote experimental reproducibility in EDA-related research to verify the reliability of existing conclusions.

Through this study, the authors call on the HCI community to address methodological weaknesses and enhance research quality, transparency, and standardization through collaborative efforts.