Sensing the Machine: Evaluating Multi-modal Interaction for Intelligent Dynamic Guidance

Recent research has demonstrated the potential for representing intelligent guidance using multi-modal cues, yet few guidelines or processes exist to guide the design of such a system. In this paper, we seek to address this gap by investigating the design of multi-modality assistant systems for setting the optimal parameters in industrial plants. We present the results of our study conducted with 22 participants to evaluate the effectiveness and experience of different combinations of visual (Highlights and Ambient lights) and haptic (Clicks and Vibration) modalities for providing intelligent dynamic guidance. Our findings demonstrate that providing the intelligent guidance with the multi modality of Highlights+Ambient resulted in shorter task duration and higher practicality than Ambient lights alone. Moreover, Highlights+Ambient+Vibration guidance was rated with lower usability than Highlights+Ambient as well as higher mental demand than merely Highlights.