Understanding Break-ability through Screen-based Affordances

Research Background and Issues

• Identified Problems or Challenges: This paper focuses on understanding the perception of "breakability" of objects through screen-based interaction. Traditional HCI (Human-Computer Interaction) research, heavily influenced by cognitivism, emphasizes reliance on internal representations (e.g., processing and predictive models in the brain). However, J.J. Gibson's ecological psychology theory of "direct perception" posits that perception is directly formed based on information in the environment, without requiring internal representations. The application of this theory in HCI research remains insufficient, particularly in screen-based interactions.

• Significance of the Issue: This "direct perception" approach, which departs from cognitivist models, helps to understand more natural interactions between humans and virtual technologies. It holds significant implications for improving screen-based interactions, enhancing immersive virtual experiences, and supporting ecological design.

• Research Motivation and Related Work:

  • Existing literature predominantly focuses on the indirect relationship between perception and internal cognitive models, with limited exploration of the direct perception theory proposed by ecological psychology.
  • In the field of interaction design, the understanding of "perceivable action possibilities" (affordances) is often confined to traditional cognitive paradigms, neglecting the ecological roots of this concept.
  • Specifically, in the domain of material perception, there is limited quantitative analysis of how "breakability" is perceived. Although multimodal perception (e.g., combining visual, tactile, and auditory cues) has recently gained attention, research on perception calibration mechanisms across technological environments remains inadequate.

Proposed Solution

• Proposed Solution: This paper introduces the "Perceptual Toughness" framework, which mathematically describes breakability in ecological perception through a novel "perceptual impact testing method" combined with a 3D virtual environment.

  • A new model based on pi-value ratios is introduced to measure breakability on an intrinsic scale (e.g., changes in subjective observation points).
  • An online 3D screen-based virtual environment is used to investigate the roles of visual and environmental factors in perception.

• Innovative Aspects of the Solution:

  1. The application of the "direct perception" theory from ecological psychology to establish a new "Ecological Physics" model for predicting perceptual behavior, rather than relying on traditional cognitivist models.
  2. Combining Gibson's "horizon-ratio" and ecological perception theories of visual angles, using mathematical models to quantify the perception of breakability.
  3. Development of a unique virtual testing environment with experimental designs incorporating independent variables (e.g., virtual observation point height and horizon geometry characteristics).

• Implementation Steps and Techniques:

  1. Mathematical Modeling: Design a formula to describe the "Critical Breaking Point" of materials, using pi-value ratios to measure subjective breakability.
  2. Virtual Platform Development: Create an online 3D virtual platform to explore the impact of external factors, such as user perspective and environmental geometry, on perception.
  3. Experimental Methods:
     • Experiment 1: Manipulate the height of the virtual observation point to evaluate whether users' judgments of breakability are systematically distorted by changes in observation points.
     • Experiment 2: Alter the geometric properties of the virtual horizon (e.g., removing implicit horizon information) to examine whether this changes users' breakability estimates.
  4. Quantitative Analysis: Evaluate users' perception using "Drop Height Error" (DHE), which represents the deviation between perceived and actual values.

Research Findings

• Specific Findings:

  • Experiment 1 (72 participants): Changes in the height of the virtual observation point caused systematic distortions in users' judgments of breakability. Higher observation points led to underestimation of breakability, while lower observation points led to overestimation.
  • Experiment 2 (66 participants): Removing or altering virtual horizon information significantly changed users' breakability estimates. Specifically, the removal of implicit horizon information resulted in a significant increase in breakability estimation errors.

• Advantages Over Existing Solutions:

  • Provides a pathway for explaining perception through intrinsic and direct mechanisms, avoiding over-reliance on internal representations compared to traditional indirect cognitive models.
  • Validates the applicability of Gibson's "direct perception" and "ecological information" (e.g., horizon and visual cues) in screen-based interactions.
  • Proposes a dynamic process of "perception calibration," which can directly quantify its impact on user behavior in virtual environments.

• Experimental/Evaluation Results:

  • Experiment 1 showed that changes in observation point height were highly significant in a linear regression analysis (p < 4.9e-06), explaining distortion effects caused by secondary information such as screen edges under extreme sample thickness conditions.
  • Experiment 2 demonstrated that removing implicit horizon information significantly increased the underestimation of breakability (p = 0.0002).

• Limitations and Future Directions:

  1. Experimental Limitations:
     • The physical observation points of participants (e.g., distance and head position) were not fully controlled. Future experiments should implement physical constraints to enhance control.
     • Screen size and resolution may have influenced the experimental results, particularly the potential role of screen edges as visual reference lines.
     • Potential confounding variables related to temporal information (e.g., perception of longer durations for greater drop heights) were not addressed.
  2. Future Research Directions:
     • Extend the study of breakability to other material properties (e.g., elasticity or toughness) to explore more complex perception models.
     • Investigate how virtual and physical observation points jointly influence the "direct perception" process.
     • Further explore how ecological calibration effects enhance user perception and behavior optimization, with potential applications in virtual gaming, driving training, and virtual reality environments.
     • Expand into the domain of dynamic interaction design (e.g., deformable devices) to optimize ecological parameters in dynamic interactions.

Conclusion

This paper demonstrates how Gibson's ecological perception theory can be faithfully applied in HCI research through the case study of "Perceptual Toughness." The proposed mathematical model and experimental validation show that even in screen-based interactions, perception remains direct and ecological. By quantifying "breakability" as a behavior-relevant feature, this study provides theoretical and methodological contributions. Future research should further explore the potential of ecological psychology in interaction design, virtual reality, and dynamic scenarios.