Exploring Reduced Feature Sets for American Sign Language Dictionaries

Research Background and Issues

• Identified Problems/Challenges:

  1. There are significant challenges in the bidirectional dictionary lookup process between American Sign Language (ASL) and English, with a lack of effective methods for finding sign language signs.
  2. Current feature-based dictionaries require complex manual input, forcing users to select appropriate descriptions from numerous features, leading to inconvenience.
  3. While video or example-based input methods are more intuitive, they rely on extensive data and computational resources, and their performance has yet to meet user needs.

• Significance:

  1. Language learners and DHH (Deaf and Hard of Hearing) users need convenient tools for language learning and reference.
  2. Enhancing the usability and accuracy of sign language dictionaries contributes to language preservation and documentation.

• Research Motivation and Related Work:

  1. This study draws on theoretical frameworks from the field of sign linguistics (e.g., linguistic theories by Stokoe) to redesign the feature set used for bidirectional dictionary searches.
  2. Current feature input methods are overly focused on complex annotation systems designed by linguists. This study aims to improve user experience by reducing and simplifying feature categories while maintaining query accuracy.

Proposed Solution

• Proposed Method:

  1. Based on two core studies: Feature Set Simplification Simulation Study and User Preference Study, this research explores reducing the feature set used for ASL-to-English dictionary queries.
  2. The approach involves reducing feature categories (e.g., removing location and orientation) and redesigning the feature sets for handshape and movement categories.

• Innovations:

  1. Proposed replacing traditional complex feature sets with a smaller-scale feature set (e.g., handshape and movement).
  2. Applied linguistic theories (e.g., Dominance and Symmetry conditions) to optimize feature selection.
  3. Employed a systematic simulation method to evaluate query accuracy and validated practical usability through extensive user testing.

• Implementation Steps and Key Techniques:

  1. Conducted "Feature Ablation Experiments" to analyze the importance of feature sets and eliminate redundant features that do not impact performance.
  2. Used Latent Semantic Analysis (LSA) to evaluate retrieval accuracy after reducing the feature set.
  3. Conducted user studies to assess the practical usability of the simplified feature set, including efficiency (input time) and usability (SUS scores).
  4. Proposed multiple reduction strategies, such as clustering handshapes based on visual similarity and substituting binary key features.

Research Outcomes

• Specific Results:

  1. After reducing feature categories to only handshape and movement features, query accuracy showed no significant decline and remained high.
  2. Proposed three reduction strategies for handshape features (e.g., clustering handshapes by visual similarity, grouping based on the number of fingers), which maintained good search performance while reducing feature complexity.
  3. User studies revealed that the reduced handshape (Clustered Handshapes) and movement features (Clustered Movements) improved the experience for some users, particularly by reducing input time.

• Advantages Over Existing Solutions:

  1. Simplified decision-making during user queries, lowering the learning curve.
  2. Provided more efficient handshape and movement feature input options, shortening input time.
  3. Achieved search performance comparable to traditional complex feature sets using fewer resources (e.g., smaller feature sets and simpler algorithms).

• Experiments and Evaluation Results:

  1. Simulation studies showed that handshape and movement features are the most critical retrieval categories, while other features (e.g., location) can be removed or simplified.
  2. User studies indicated that different users preferred different simplified feature schemes. Some users favored simpler feature groupings, while those familiar with traditional dictionaries preferred the more complex but familiar feature sets.
  3. For complex signs involving multiple handshapes or movements, users suggested improvements such as interfaces supporting multi-stage input.

• Limitations and Future Directions:

  1. Limitations:
     • The dataset size covered only 1,145 ASL signs, which is not representative of the entire dictionary vocabulary.
     • The user study sample was biased toward ASL learners (non-experts or non-native users).
     • The feature set used in the study was based on the existing Stokoe system, potentially limiting the diversity of feature design.
  2. Future Directions:
     • Extend the research to larger vocabularies or real-world usage scenarios.
     • Explore feature simplification strategies for other sign languages or cross-cultural sign language dictionaries.
     • Design more modular interfaces that allow users to customize feature sets.
     • Introduce auto-completion or search interface optimizations closer to real-world scenarios.

This study provides valuable insights for the future design of sign language dictionaries, not only simplifying query feature sets but also focusing on user experience, demonstrating the potential for large-scale applications of ASL dictionaries.