Multifractal Mice: Inferring Task Engagement and Dimensions of Readiness-to-hand from Hand Movement

Title of the Paper

Multifractal Mice: Operationalising Dimensions of Readiness-to-hand via a Feature of Hand Movement

Paper Information

• Research Domain: Human-Computer Interaction (HCI), Cognitive Science and Philosophical Studies of Tool Use
• Keywords: readiness-to-hand, embodiment, phenomenology, cognitive science, complex systems, engagement, user experience

Research Background and Problem Statement

• Identified Problems or Challenges:

  1. The philosophical concept of "readiness-to-hand" (intuitive and skillful tool use) has a long research history but lacks measurable and experimentally validated operational indicators.
  2. Existing studies attempting indirect measurement of readiness-to-hand often rely on highly intrusive secondary tasks, which may hinder natural tool use.
  3. Most current research focuses on tool breakdown scenarios, neglecting other factors that may influence readiness-to-hand, such as familiarity and task engagement.

• Significance: Readiness-to-hand is considered the essence of ideal human-computer interaction experiences. Operationalizing its measurement can contribute to designing natural and seamless technological experiences, making designs more intuitive and aligned with users' actual needs.

• Research Motivation: Drawing on the concept of multifractality from cognitive science, the authors hypothesize that this nonlinear dynamic property is related to certain behavioral and experiential dimensions of readiness-to-hand. The study aims to address the gap in measurement dimensions.

Solution

• Proposed Methods or Solutions:

  1. Apply multifractal analysis (MFA) to hand movements, quantifying readiness-to-hand through dynamic complexity indicators such as multifractal spectrum width.
  2. Explore the relationship between multifractality and tool familiarity, breakdown, and task engagement through two experiments.
  3. Develop new parameter adjustment methods to optimize hypothesis testing in multifractal analysis while reducing errors caused by linear factors in the analysis.

• Innovations:

  1. Introduced a non-intrusive, hand-movement-based quantification method, making the measurement of readiness-to-hand closer to natural interaction scenarios.
  2. Expanded readiness-to-hand research to include dimensions of task engagement and familiarity.
  3. Provided a data-driven parameter selection method, verifying the reliability of multifractal spectra through surrogate analysis.

• Implementation Steps:

  1. Experiment 1: Test whether multifractality reflects changes in user attention during tool familiarity and tool breakdown scenarios using a mouse-based game task (N=44).
  2. Experiment 2: Compare the effects of two games with different levels of engagement (high engagement vs. low engagement) on multifractal spectrum width, while observing the impact of task duration on multifractality (N=30).
  3. Clean hand movement data, extract multifractal features using Wavelet Transform Modulus Maxima (WTMM) analysis, and validate the reliability of the analysis through surrogate analysis.

Research Findings

Key Discoveries

1. Tool Breakdown and Attention Shift:

   • Tool breakdown significantly increases users' attention to the tool, with no change in attention to non-task-related elements.
   • Hand movements in breakdown scenarios exhibit lower multifractal characteristics, confirming the fractal correlation of readiness-to-hand states.

2. Familiarity and Readiness-to-hand:

   • After multiple rounds of tasks, users familiarized with the tool showed a significant increase in multifractal spectrum width, indicating that readiness-to-hand improvement is related to familiarity.

3. Task Engagement:

   • High-engagement tasks (achieved through challenging design and real-time feedback) resulted in higher multifractal spectrum width.
   • During task duration, all participants showed a slight decrease in spectrum width in later stages, possibly reflecting fatigue or reduced attention.

4. Methodological Validation:

   • The proposed parameter selection method for multifractal analysis was validated through surrogate analysis, effectively eliminating interference from linear factors in multifractal measurement.

Advantages Over Existing Research

• Eliminates the need for intrusive secondary tasks, making the experimental environment closer to actual interaction scenarios.
• Expands the measurement scope of readiness-to-hand to include tool familiarity and task engagement.
• Provides a more reliable multifractal spectrum validation framework, enhancing the theoretical interpretability of the findings.

Limitations and Future Directions

1. Limited Experimental Tasks:

   • The experimental tasks were confined to mouse-based games, and the applicability of multifractality in other interaction modalities (e.g., keyboard use, VR/AR, natural gesture interaction) remains untested.

2. Distinguishing Fatigue and Interest:

   • The decrease in fractality observed in Experiment 2 may be influenced by fatigue or reduced interest. Further variable separation is needed to clarify causal relationships.

3. Association with Subjective Experience:

   • The experiments primarily focused on behavioral data, without detailed exploration of how readiness-to-hand affects first-person experiences (e.g., immersion and responsiveness).

4. Challenges in Real-time Application:

   • Although multifractal methods are theoretically applicable in real-time environments, their real-time performance and response latency need empirical testing and optimization.

Open Data and Tools

• All experimental data, analysis scripts, and method parameter settings are openly available on OSF Repository.