Accessible Data Representation with Natural Sound

Document Title

Accessible Data Representation with Natural Sound

Document Information

• Subject Area: Data visualization and accessible design, particularly sonification techniques for blind and low-vision (BLV) users
• Keywords: Sonification, accessibility, data visualization, natural sound, user study, data perception, inclusive design

Research Background and Issues

• Identified Problem or Challenge: The visual nature of data visualization makes it difficult for blind and low-vision (BLV) users to effectively access such information. Traditional sonification techniques (e.g., using artificially generated notes or music) face multiple issues, such as difficulty perceiving multiple data points, distraction, and challenges in sound recognition. These methods are rarely deployed in commercial software.
• Significance: Digital visualization has become a critical method of information presentation in daily life, with applications ranging from data science to healthcare. The inability to access this information poses significant barriers for BLV users, restricting their equitable access to information.
• Research Motivation: The authors hypothesize that natural sounds (e.g., bird chirps, raindrops) can more effectively convey data due to their familiarity, comfort, and ability to support parallel perception, particularly for users without a musical background.

Solution

• Method or Solution: The authors propose a natural sound-based sonification tool called Susurrus, which converts common data visualizations (e.g., bar charts, line graphs, scatter plots) into natural sound formats.
• Innovations:
  • Utilizing natural sounds as the core medium for sonification, leveraging their emotional connection and familiarity.
  • Supporting parallel playback of sounds (simultaneous sound effects) instead of traditional sequential playback.
  • Implementing techniques based on LUFS (Loudness Units relative to Full Scale) and keyboard interactions to provide precise loudness mapping and user-friendly controls.
• Implementation Steps and Key Technologies:
  1. Compiled a sound effect library, including natural environmental sounds such as bird chirps, wave sounds, and wind noises.
  2. Normalized sound loudness using the LUFS algorithm and mapped loudness to data values.
  3. Enabled parallel playback, allowing users to perceive multiple data points simultaneously.
  4. Implemented AISA (Auditory Information Search Actions) via keyboard interactions, such as navigation, selection, and detail requests.
  5. Developed a web-based tool compatible with screen readers and accessible browsers.

Research Outcomes

• Specific Results:
  • Natural sounds effectively support common data operations, such as bar charts, line graphs, and scatter plots.
  • Compared to existing sonification tools (e.g., Highcharts), Susurrus is better suited for representing multi-category data, especially bar charts.
  • Non-musical background users demonstrated significantly higher accuracy when using Susurrus compared to existing sonification tools.
  • Experiments showed that natural sounds have hedonic value, helping to relax emotions, enhance focus, and personalize the user experience.
• Experimental or Evaluation Results:
  • In bar chart tasks, Susurrus achieved approximately 9.7% higher accuracy than Highcharts.
  • In line graph tasks, Susurrus performed comparably to Highcharts but was more effective for users without a musical background.
  • Using NASA-TLX and SUS metrics, users rated the tool highly in terms of workload and usability.
• Limitations and Future Directions:
  • Susurrus currently supports only bar charts, line graphs, and scatter plots. Future plans include expanding to other chart types, such as pie charts and node-link diagrams.
  • Parallel sonification is limited to a small number of data categories (recommended maximum of five). Future improvements may involve hierarchical or data filtering methods to address this limitation.
  • Challenges in sound mixing: Enhancements in sound processing technology are needed to improve playback quality and aesthetic appeal.
  • Exploring the potential of natural sounds in immersive and multimodal data presentations, such as panoramic data visualization in virtual reality (VR).

Contributions and Conclusion

• Contributions:
  • Pioneered the use of natural sounds in data visualization sonification design.
  • Proposed Susurrus, a tool supporting natural sound conversion for multiple data chart types.
  • Provided user study evidence demonstrating the effectiveness of natural sound sonification, particularly for non-musical background users.
• Conclusion: Susurrus breaks the paradigm of traditional sonification, showcasing the potential of natural sounds in accessible data design. It offers new methods and directions for future inclusive data representation.