A Contextual Inquiry of People with Vision Impairments in Cooking

Title of the Paper

A Contextual Inquiry of People with Vision Impairments in Cooking

Paper Information

• Subject Area: Accessibility Technology and Human-Computer Interaction
• Keywords: Cooking, Contextual Inquiry, Blindness, People with Vision Impairments, Accessible Design, Assistive Technology, AI Systems

Research Background and Problem

Issues or Challenges

• Visual cues are a critical source of information during cooking, but people with vision impairments cannot rely on these cues, leading to numerous challenges in cooking and food preparation, such as locating ingredients and determining whether food is fully cooked.
• Existing visual assistance systems (e.g., SeeingAI and TapTapSee) have object recognition capabilities but perform poorly in kitchen environments, struggling to identify complex object groups or provide precise information.
• A lack of systematic research on the specific contextual information needs of people with vision impairments in cooking and the underlying reasons for these needs.

Importance

• Acquiring contextual information about kitchen objects through non-visual means has a significant impact on the independence and quality of life of people with vision impairments.
• A deeper understanding of these needs and behaviors can guide the design of more effective assistive technologies, reducing barriers and enhancing independence.

Research Questions

• RQ1: What contextual information about objects is important for people with vision impairments during cooking?
• RQ2: What non-visual techniques and strategies do people with vision impairments use to acquire and utilize object information?
• RQ3: What are the most effective methods for delivering necessary contextual information about objects at the right time and in the right way?

Solutions

Methods or Design

1. Contextual Inquiry Design:

   • Participant Selection: The study recruited 12 participants with cooking experience and vision impairments, distributed across different household settings (living alone, with family, etc.).
   • Study Process: Included pre-interviews (to understand participants' backgrounds), in-situ contextual inquiries (recording participants' cooking behaviors in their kitchens), and semi-structured interviews (exploring participants' needs and opinions on assistive technology).
   • Contextual inquiries were documented using wearable cameras and static cameras to capture participants' activities.

2. Data Analysis:

   • Thematic analysis was used to code video and interview data, focusing on contextual information needs, information acquisition processes, and preferred communication methods.

Innovations

• Defined a systematic classification of kitchen object contextual information, including primary attributes (e.g., location, orientation), secondary attributes (e.g., safety), and application-specific attributes (e.g., plating).
• Systematically recorded how people with vision impairments use multi-sensory methods such as touch, hearing, and smell to acquire this information.
• Proposed clear communication methods and design considerations for information delivery.

Implementation Steps

• Observed participants performing "natural cooking" and specific tasks (e.g., identifying kitchen objects, organizing space) in their own kitchens during contextual inquiries.
• Conducted semi-structured interviews to understand their motivations and preferences for information communication methods.
• Used multi-sensor data to document the complete process from behavior to information needs.

Research Findings

Specific Findings

1. Classification of Kitchen Object Contextual Information:

   • Primary Attributes:
     • Location: The position of objects relative to reference points.
     • Orientation: The alignment of objects relative to reference points.
     • Proximity and Grouping: The spatial arrangement of objects with respect to others.
     • Similarity and Repetition: Differentiating and locating similar or repeated objects.
     • Internal State: Attributes such as cleanliness, temperature, freshness, and the internal state of objects.
   • Secondary and Application-Specific Attributes:
     • Safety-related information (e.g., preventing items from falling or monitoring flame hazards).
     • Health-related information (e.g., food expiration, overcooking).
     • Plating and presentation information (e.g., color distribution, food arrangement).

2. Methods and Strategies for Acquiring Contextual Information:

   • Multi-Sensory Usage:
     • Using touch to scan the position and state of objects (e.g., food freshness).
     • Using hearing to perceive the quantity of liquid flow and cooking status.
     • Using smell to judge whether food has spoiled or is cooked.
   • Actively Creating Additional Contextual Information: Using uniquely shaped containers, spatial grouping, and object fixation.
   • Optimizing Cooking Processes: Pre-arranging items, simplifying steps, and reducing precise spatial actions.

3. Information Communication Needs:

   • More precise spatial descriptions (e.g., based on kitchen reference points).
   • Limiting verbose information to avoid excessive interruptions.
   • Multi-modal interaction: Combining verbal and non-verbal feedback.

Advantages

• Fills a knowledge gap in academic research regarding the cooking behaviors and contextual information needs of people with vision impairments.
• Provides actionable recommendations for designing assistive technologies, such as how AI assistants can effectively convey object information.

Experimental or Evaluation Results

• Contextual inquiries revealed that people with vision impairments use complex multi-layered behaviors to acquire information but face significant efficiency and safety challenges.
• Participants provided diverse suggestions regarding the types and delivery methods of assistive technology.

Limitations and Future Directions

• Limitations:

  • Participants were limited to the Chinese region, leaving the needs of people with vision impairments in other cultural contexts unexplored.
  • All participants had cooking experience, leaving the needs of beginners or those with no experience unaddressed.

• Future Directions:

  • Expand the study to include international user groups and diverse cultural environments.
  • Explore low-cost and portable smart tags or assistive devices.
  • Develop home AI systems with environmental recognition capabilities for more complex indoor navigation and contextual tracking.

The above summary comprehensively reflects the structure, methods, and findings of the paper, making it suitable for in-depth understanding or further related research.