MaGEL: A Soft, Transparent Input Device Enabling Deformation Gesture Recognition

DOI

We propose MaGEL, a soft-input device that utilizes light intensity to detect and interpret user deformation interactions. Unlike traditional rigid input devices, MaGEL enables three-dimensional interactions such as twisting, bending, and pulling. Additionally, MaGEL incorporates elastic haptic feedback, providing users with tactile sensations that reflect the tension or resistance of their interactions. These factors realize intuitive and natural user interaction experiences, and users can employ familiar physical gestures as input. For example, bending the device may simulate turning the page of a book, or stretching it may zoom in on an image. The device consists of a transparent urethane resin gel with LED lights and phototransistors on both sides. When the device gel deforms, the intensity of the light passing through the gel undergoes a specific change due to the deformation. The system analyzes these changes using machine learning to identify the user gestures. We evaluated the optimal configuration and number of LEDs and phototransistors to classify the deformation accurately. We acquired data for 13 types of deformation gestures from 14 participants. The results showed that a combination of four LEDs and ten phototransistors enabled MaGEL to identify 13 types of deformation gestures with an accuracy of 94.1 %. Using MaGEL, we provide novel interactive experiences, such as game controllers that employ bending, pulling, or twisting to mimic natural gaming motions.