Voice-Based Chatbots for English Speaking Practice in Multilingual Low-Resource Indian Schools: A Multi-Stakeholder Study

Paper Title

Voice-Based Chatbots for English Speaking Practice in Multilingual Low-Resource Indian Schools: A Multi-Stakeholder Study

Publication Info

• Topic area: Deployment and evaluation of voice-based chatbots for English language learning in low-resource educational settings.
• Keywords: Voice-based chatbot, English language learning, multilingual classrooms, low-resource schools, educational technology, conversational AI, speech recognition, student engagement, teacher analytics, HCI4D.

Background and Problem

• Problem / challenge: Many low-resource Indian schools lack opportunities for spoken English practice due to large class sizes, limited teacher proficiency, and infrastructural deficits. Existing EdTech tools often fail to address these challenges effectively in multilingual, low-connectivity contexts.
• Significance: Spoken English proficiency is a key driver of economic mobility in India, particularly for youth entering customer-facing and service roles. Addressing the gap in spoken English practice can improve confidence and future opportunities for students.
• Motivation and related work: Prior studies show the potential of conversational agents in language learning but highlight challenges like speech recognition accuracy and feedback quality, especially in low-resource, multilingual settings. This study builds on these insights to explore the feasibility and design considerations for deploying voice-based chatbots in Indian schools.

Solution

• Proposed approach: Deployment of a prototype voice-based chatbot, ChatFriend, designed to provide conversational English practice for students in low-resource schools.
• Novelty:
  1. A six-day, multi-stakeholder field study involving students, teachers, and principals in four low-fee Delhi schools.
  2. Identification of design tensions between open-ended conversational practice and curriculum-aligned assessment.
  3. Development of design recommendations for voice-based educational technologies in multilingual, resource-constrained contexts.
  4. Insights into technical and pedagogical factors affecting chatbot usability and engagement.
• Procedure and key techniques:
  • ChatFriend facilitated voice-first interactions on everyday topics using a hold-to-talk interface.
  • Speech was transcribed using Whisper-1 and processed by GPT-4o-mini, with responses moderated for safety and synthesized into speech.
  • Students participated in supervised and unsupervised sessions, and qualitative data were collected through observations, interviews, and feedback.

Results

• Concrete findings:
  • Students’ confidence in speaking English increased over time, with 76.5% predominantly speaking English by Day 5 (up from 29.4% on Day 1).
  • Students produced an average of 5 tokens per turn, compared to the chatbot’s 29 tokens, highlighting a gap in linguistic complexity.
  • Technical issues like ASR errors and network latency disrupted engagement, with 35% of students requiring guidance to use the microphone interface.
• Advantage over baselines:
  • ChatFriend provided a non-judgmental, low-stakes environment for practice, addressing students’ speaking anxieties more effectively than traditional classroom methods.
• Experiments / evaluation:
  • Conducted in four low-fee private schools with 23 students, 6 teachers, and 5 principals.
  • Data collected through real-time observations, interviews, and session-level affect ratings.
  • Affective trajectories revealed diverse patterns, including confidence surges and frustration arcs.
• Limitations and future work:
  • Short intervention period (six days) limits insights into long-term engagement and learning outcomes.
  • Lack of pre- and post-interaction assessments prevents definitive claims about learning gains.
  • Future work should include longitudinal studies, standardized assessments, and technical refinements like accent-tuned ASR.

Summary

This study evaluated the deployment of a voice-based chatbot, ChatFriend, for English speaking practice in four low-resource Delhi schools. The chatbot increased students’ confidence and provided a safe, non-judgmental space for language practice, though technical issues like ASR errors and network latency posed challenges. Teachers and principals valued the chatbot’s potential for assessment and curriculum alignment, while students preferred open-ended conversational practice. The study highlights design priorities, including simplified interfaces, accent-tuned speech recognition, and teacher-facing analytics, for scaling voice-based educational technologies in resource-constrained, multilingual contexts. Future work should focus on long-term efficacy, technical improvements, and integration into school systems.