Cultivating Spoken Language Technologies for Unwritten Languages

Title of the Paper

Cultivating Spoken Language Technologies for Unwritten Languages

Paper Information

• Subject Areas: Human-Computer Interaction, Natural Language Processing, Linguistics, Design Research
• Keywords: Spoken Language, Zero-Resource Information Retrieval, Co-Creation, Field Research, Participatory Design, Agricultural Culture, Digital Divide, Unwritten Languages

Research Background and Issues

Challenges Identified by the Authors

• Of the approximately 7,000 languages globally, 40% are endangered and may become extinct by 2050. The vast majority of these languages lack digital resources or support.
• Current digital user interfaces and information retrieval methods are predominantly based on written cultures, with limited adaptation for communities that primarily rely on oral languages.
• ICT4D (Information and Communication Technology for Development) and innovations in speech technologies hold significant potential for serving marginalized communities, yet resources for unwritten languages remain scarce.

Importance of the Research

• Innovations in cross-linguistic and speech technologies can empower communities in the Global South and preserve endangered languages, bridging the digital divide.
• Technologies supporting oral languages can facilitate the digital preservation and enhancement of cultural practices, complementing traditional methods of language documentation and preservation.

Motivation and Related Work

• The authors conducted participatory research in a community in Maharashtra, India, that uses the unwritten Gormati language, aiming to develop an accessible speech-driven information retrieval system.
• Current technologies and practices (e.g., traditional IVR and IVF platforms) often impose written language hierarchies when addressing oral communities, which may not align with the needs of unwritten languages.
• The paper discusses Orality Theory and its potential to inspire user interface design, focusing on the impact of oral culture and cognitive structures on technology adaptation.

Solutions

Proposed Approach

• Participatory Design: Collaborating with the Banjaras community in India, integrating HCI, linguistics, and NLP technologies.
• Information Retrieval System: A speech-based retrieval system utilizing phonetic recognition and ranking models for cross-speech matching.
• Social Integration: Conducting fieldwork, data collection, and technology development in phases, with prototype applications evaluated and co-created locally.

Innovations in the Approach

1. Decentralized Development: Starting directly with oral data contributed by the community, without relying on existing written or digital resources.
2. Iterative Participatory Development: Low data requirements, needing only 4 hours of speech data, with gradual optimization as the dataset expands.
3. Sensitive Data Handling: Avoiding written transcription to respect and adapt to the community's oral practices and cultural structures.

Implementation Steps and Key Technologies

1. Fieldwork and Immersion: Conducting three visits to deeply understand the agricultural community and its linguistic and cultural frameworks.
2. Data Collection and Technology Development:
   • Collecting Gormati speech data through photo annotation tasks.
   • Developing the information retrieval system using Support Vector Machines (SVM) and cross-linguistic phoneme recognition models.
3. Testing and Evaluation: Deploying an Android prototype application for formal evaluation within the community.
4. Future Development: Supporting continuous data collection via familiar platforms like WhatsApp to train and refine algorithms.

Research Outcomes

Specific Results

• Information Retrieval Accuracy: The proposed system successfully returned the target photo as one of the top 5 results in 74% of test cases.
• Validation of Design Methods: Demonstrated the feasibility of developing speech technologies tailored to oral cultures and low-resource languages.

Advantages Over Existing Solutions

• Cultural Alignment: Avoids imposing written language structures on the community, directly supporting oral languages.
• Resource Sustainability: Does not rely on large-scale datasets or complex technologies.
• Broad Impact: Scalable to other oral culture communities, not limited to a single region or language.

Experiment and Evaluation Results

• Community members naturally used oral descriptions to query target content, rather than relying on keywords or specific phrases.
• Feedback from real-world community use included customized use cases such as digital agricultural techniques, song sharing, and cooking videos.

Limitations and Future Directions

• Data Scarcity: The system's capabilities are constrained by the available data, and the photo annotation task did not sufficiently motivate sustained community participation.
• Cross-Cultural Adaptability: Despite multiple field tests, the method's applicability to other languages and regions remains to be validated.
• Next Development Steps: Building a community-based tablet system for media storage and access; designing more advanced information retrieval tasks for complex content such as songs and videos.

Conclusion

• This paper proposes a feasible approach to designing speech technologies for unwritten languages from scratch, tailored to communities with oral cultural backgrounds.
• By integrating multidisciplinary methods (HCI, NLP) with action research, the authors successfully developed a speech-driven information retrieval system for low-resource languages. The approach is built on community collaboration and sustainable data collection methods, addressing the unique characteristics of the target language and culture.
• Future work will focus on iterative development and expansion to achieve broader and deeper community engagement and application of the technology.

For further in-depth interpretation and citation, please refer to the original paper.