ProVoice: Designing Proactive Functionality for In-Vehicle Conversational Assistants using Multi-Objective Bayesian Optimization to Enhance Driver Experience

Paper Title

ProVoice: Designing Proactive Functionality for In-Vehicle Conversational Assistants using Multi-Objective Bayesian Optimization to Enhance Driver Experience

Publication Info

• Topic area: Proactive intervention design for in-vehicle conversational assistants (IVCAs) using optimization techniques.
• Keywords: Proactive IVCA, Multi-Objective Bayesian Optimization, Human-in-the-Loop, Driver-Vehicle Interaction, Virtual Reality, Level of Autonomy, Predictability, Usefulness, Mental Demand, Personalization.

Background and Problem

• Problem / challenge: Current IVCAs are predominantly reactive, requiring user activation, and fail to address diverse driver preferences and dynamic driving contexts. Designing proactive interventions involves balancing multiple opaque and subjective objectives like trust, usefulness, and mental demand.
• Significance: Proactive IVCAs could enhance safety, convenience, and engagement by initiating timely, context-aware interactions, but require systematic approaches to personalization.
• Motivation and related work: Previous studies explored reactive IVCA designs and user-led personalization, but these approaches often overlook optimal configurations and fail to address multi-objective trade-offs. This paper builds on computational design techniques like HITL MOBO to explore proactive IVCA design.

Solution

• Proposed approach: ProVoice, a VR driving simulator leveraging Human-in-the-Loop Multi-Objective Bayesian Optimization (HITL MOBO) to optimize proactive IVCA designs across multiple parameters and objectives.
• Novelty:
  1. Development of ProVoice, a VR simulator for proactive IVCA design, integrating HITL MOBO.
  2. Empirical evaluation of proactive IVCA designs through a within-subjects study with 19 participants.
  3. Open-source release of Unity assets for proactive IVCA modeling and optimization.
  4. Exploration of Level of Autonomy (LoA) as a design parameter, with comparisons between fixed and trained LoA.
• Procedure and key techniques:
  • Design parameters: Interior lighting glow, auditory alert volume, symbol transparency, and LoA (5 levels).
  • Objectives: Minimize mental demand; maximize predictability and usefulness.
  • HITL MOBO setup: Gaussian-process-based optimization using q-Noisy Expected Hypervolume Improvement (qNEHVI) acquisition function.
  • Study design: Participants completed two conditions (Trained LoA and Fixed LoA) in a VR driving simulator, providing iterative feedback for optimization.

Results

• Concrete findings:
  • HITL MOBO reduced mental demand and increased predictability and usefulness across iterations.
  • Predictability and usefulness were strongly positively correlated (r = 0.47), while mental demand and usefulness were negatively correlated (r = -0.32).
  • Trained LoA required more iterations to align with driver expectations compared to Fixed LoA, which started with higher baseline scores.
• Advantage over baselines:
  • HITL MOBO systematically explored design spaces, uncovering Pareto-optimal configurations that user-led approaches might overlook.
  • Fixed LoA aligned with drivers’ proactive dispositions, achieving consistently high predictability and usefulness scores.
• Experiments / evaluation:
  • Participants: N=19 (10 male, 9 female, Mean Age = 31.4).
  • Procedure: Within-subjects VR study with two conditions (Trained LoA, Fixed LoA), optimizing IVCA designs over 12–20 iterations.
  • Metrics: NASA TLX for mental demand, trust in automation survey for predictability, and USE questionnaire for usefulness.
• Limitations and future work:
  • Limited sample size and potential biases due to opportunity sampling.
  • Cold-start MOBO approach; future work could explore warm-start methods.
  • Need for real-world evaluations in dynamic driving conditions.
  • Privacy concerns regarding data collection for proactive functionality.

Summary

This paper introduces ProVoice, a VR-based driving simulator that uses HITL MOBO to personalize proactive IVCA designs. The study demonstrates that HITL MOBO effectively reduces mental demand and increases predictability and usefulness, with Fixed LoA aligning better with driver expectations than Trained LoA. The open-source Unity assets enable broader exploration of proactive IVCA designs. Future work should address privacy concerns, explore alternative optimization methods, and validate findings in real-world driving scenarios.