GesturePod: Enabling On-device Gesture-based Interaction for White Cane Users

People using white canes for navigation find it challenging to concurrently access devices such as smartphones. Building on recent research on abandonment of specialized devices, we explore a new touch free mode of interaction, wherein a person with visual impairment performs gestures on their existing white cane to trigger tasks on their smartphone. We present an easy-to-integrate GesturePod, that clips on to any white cane and enables the detection of gestures performed with the cane. GesturePod, thereby, helps perform certain tasks on a smartphone without touch, or removing the phone from a pocket or bag. We discuss challenges that arise in building the pod and design decisions we made. We propose a novel, efficient machine learning pipeline to train and deploy a gesture recognition model. Our in-lab study shows that GesturePod achieves 92% gesture-recognition accuracy and can help perform some common smartphone tasks faster. Our in-wild study suggests that GesturePod is a promising interaction tool for smartphone, especially in constrained outdoor scenarios.