Ellie Talks About the Weather: Toward Evaluating the Expressive and Enrichment Potential of a Tablet-Based Speech Board in a single Goffin’s Cockatoo

Title of the Paper

Ellie Talks About the Weather: Toward Evaluating the Expressive and Enrichment Potential of a Tablet-Based Speech Board in a Single Goffin’s Cockatoo

Paper Information

• Research Area: Animal-Computer Interaction (ACI), Augmentative and Alternative Communication (AAC), Animal Cognition and Behavior Studies
• Keywords: Animal-computer interaction, touchscreen interaction, animal enrichment, avian learning, Goffin’s cockatoo, social cognition, augmentative communication, technology and animals, animal behavior, human-animal relationships

Research Background and Problem

• Challenges:

  1. Scientific research on animals using AAC devices for communication is scarce, particularly lacking systematic evaluation frameworks in the animal domain.
  2. There is no consistent method to verify animals' intent in using AAC devices or to analyze their behavior.
  3. Existing AAC devices are primarily designed for humans and are not fully adapted for animal use.
  4. Long-term studies on animal interaction with digital AAC devices are rare.

• Significance: This study can deepen the understanding of communication between humans and animals and potentially improve animal enrichment through technology, enhancing their mental health, quality of life, and sense of environmental control.

• Motivation and Related Work: Research indicates that AAC devices have the potential to provide social and cognitive enrichment for animals, enhancing their communication abilities. However, further development of validation frameworks and data collection is necessary. This study focuses on a long-term case of a Goffin’s cockatoo using a tablet-based AAC device, aiming to evaluate the expressive and enrichment value behind its behavior.

Solution

• Methods and Approach:

  1. Experimental Setup: Data were collected from a Goffin’s cockatoo named Ellie over a 7-month period, encompassing 39 hours of interaction video across 129 independent experimental days.
  2. Framework Design: A novel data coding and annotation framework was developed to meticulously record the cockatoo's interaction behaviors with the device, environmental context, and interactions with human caregivers.
  3. Data Analysis: A random modeling approach was employed to compare the data against simulated random selection behaviors, along with categorical analysis of the cockatoo’s behavior to identify thematic distributions.

• Innovations:

  1. Developed a specialized classification and coding framework to comprehensively record and quantify animal interactions with AAC devices.
  2. Combined traditional human AAC evaluation frameworks with animal behavior studies to explore the enrichment functionality of augmentative communication devices.
  3. Systematically quantified the expressive potential of such devices through random modeling, interaction validation, and thematic classification analysis.

• Implementation Steps and Techniques:

  1. Documented details of each experiment (location, time, behavior, interaction) and stored relevant video recordings.
  2. Created a tree-structured dataset corresponding to the device interface and designed a random traversal simulator to generate baseline data.
  3. Used Generalized Linear Mixed Effects Models (GLMM) to evaluate behavioral preferences and deviations from randomness.
  4. Verified the validity of the cockatoo’s expressions through semantic classification and behavioral validation (e.g., time investment, task completion).

Research Findings

• Specific Results:

  1. Non-random Selection Behavior: The cockatoo’s selection of specific words could not be explained solely by the randomness of the device interface; multiple model comparisons indicated deviations from random choice behavior.
  2. High Validation Rate of Selection Behavior: Among verifiable behaviors, 92% were confirmed (via task completion or behavioral engagement).
  3. Thematic Distribution of Choices:
     • Selections related to “social-cognitive interaction” dominated (72.5%).
     • Compared to the proportional distribution of icons on the AAC device, food-related choices were proportionally similar, while environmental information choices were significantly underrepresented.

• Advantages Over Existing Solutions:

  1. Long-term, Systematic Data Collection: The study spans 7 months, making it a rare long-duration experimental study in the field of animal AAC.
  2. Dual Exploration of Expressiveness and Enrichment: The research not only focuses on communication expression but also evaluates enrichment value, contributing to animal welfare.
  3. Methodological Contributions: Provides a comprehensive annotation and evaluation framework applicable to future studies on other species and individuals.

• Limitations and Future Directions:

  1. Limitations:
     • The study involves a single subject, limiting generalizability.
     • The large volume of data means current analysis does not cover all interaction scenarios.
     • Technical issues (e.g., device malfunctions) may impact data completeness.
  2. Future Directions:
     • Expand research to include multiple cockatoos and other animals to validate the generalizability of the methods.
     • Conduct long-term tracking to analyze behavioral patterns and continuously refine annotation and evaluation frameworks.
     • Design AAC devices specifically tailored for animal users, incorporating higher levels of personalization and reliability.