Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool

Paper Title

Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool

Publication Info

• Topic area: Accessibility in 3D data visualization for blind and low-vision (BLV) users.
• Keywords: 3D visualization, accessibility, blind and low-vision, sonification, tactile prototypes, multi-modal interaction, Experience-Based Co-Design, spatial audio, STEM data, embodied cognition.

Background and Problem

• Problem / challenge: Existing 3D data visualization tools are largely inaccessible to BLV individuals, limiting their ability to engage with spatially complex datasets critical to STEM fields.
• Significance: Making 3D visualizations accessible would enable BLV users to participate more fully in scientific analysis and knowledge production, addressing equity gaps in STEM disciplines.
• Motivation and related work: Prior accessibility efforts have focused on 2D tactile graphics and audiotactile approaches, with limited exploration of 3D visualizations. Existing methods often fail to preserve spatial topology or provide pedagogical scaffolding for independent exploration. This study builds on the Experience-Based Co-Design (EBCD) framework to address these gaps.

Solution

• Proposed approach: A web-native, multi-modal 3D visualization tool co-designed with BLV collaborators, integrating tactile prototypes and digital features such as sonification, spatial audio, and configurable buffer aggregation.
• Novelty:
  1. Transferable co-design protocol for translating tactile knowledge into digital interactions.
  2. Validated prototype enabling independent non-visual exploration of 3D data.
  3. Empirically grounded design principles for accessible 3D visualization.
  4. Exploration of embodied and immersive interfaces for future development.
• Procedure and key techniques:
  • Phase 1: Foundational ideation with BLV collaborators to create an initial high-fidelity digital probe.
  • Phase 2: Iterative refinement through tactile probe fabrication, digital integration, and two co-design sessions.
  • Implemented features include reference sonification, stereo and volumetric audio, and configurable buffer playback.

Results

• Concrete findings:
  • Reference sonification improved spatial orientation (validated by co-designers).
  • Stereo panning effectively conveyed horizontal position; depth perception required reverb augmentation.
  • Configurable buffer playback supported regional comparison but revealed usability issues (e.g., selection initiation flow, boundary awareness).
• Advantage over baselines:
  • Provides scalable, low-cost access to 3D data without specialized hardware.
  • Supports core analytic tasks (e.g., gradient tracing, peak finding) through multimodal interaction.
• Experiments / evaluation:
  • Two co-design sessions with six BLV collaborators.
  • Tested features through task-based prompts targeting analytic workflows like 3D orientation and region comparison.
  • Validation focused on dense, continuous datasets (e.g., benzene spectroscopy, Gaussian surfaces).
• Limitations and future work:
  • Expert sample bias; findings may not generalize to novice BLV users.
  • Limited dataset types; sparse or noisy data remains untested.
  • Cross-sectional study design; longitudinal effects (e.g., auditory fatigue) not assessed.
  • Planned refinements include reverb for depth perception, alternative input devices, and comparative plot views.

Summary

This study presents a web-native, multi-modal 3D visualization tool co-designed with BLV collaborators to address accessibility gaps in spatial data analysis. Using tactile prototypes and iterative feedback, the tool integrates features like sonification, spatial audio, and configurable buffer playback to support analytic tasks such as gradient tracing and peak finding. Validation demonstrated improved orientation and analytic accuracy, though usability issues remain. The work contributes a transferable co-design protocol, empirically grounded design principles, and a pathway toward embodied and immersive interfaces for future accessible visualization systems.