GestApt: A Pen-Based Interface Integrating Gestures and Recommendations for CAD Tasks

Pen-based interaction interfaces are widely used in precision tasks such as computer-aided design (CAD) modeling. However, frequent mode switching in traditional interfaces often leads to reduced efficiency, while the unstable layout of dynamically adaptive interfaces may cause user discomfort. To address these issues, this paper proposes a novel adaptive pen-based interface, GestApt, which integrates operation recommendation and pen gesture technology to provide an efficient and intuitive interaction method. The core design of GestApt includes a gated recurrent unit (GRU)-based neural network for a predictive toolbar that dynamically forecasts the user’s next action and a convolutional neural network (CNN)-based gesture recognition module to assist with unpredicted operations. To evaluate the effectiveness of GestApt, we conducted a user study with 20 participants performing CAD modeling tasks, comparing their performance with a traditional interface. The results showed that GestApt significantly reduced task completion time while enhancing user experience. Additionally, the optimized toolbar design of GestApt notably reduced the workload on users’ non-dominant hands, helping to alleviate physical fatigue from prolonged interface use. This work demonstrates the advantages of GestApt in improving efficiency, optimizing user interaction, and reducing physical strain, offering a new solution for adaptive design in pen-based interfaces. This design approach not only promotes the development of more efficient and ergonomically friendly interfaces, but also provides a reference for the design of future intelligent CAD interfaces.