A Dialogue-Based Interface for Active Learning of Activities of Daily Living

Title of the Paper

A Dialogue-Based Interface for Active Learning of Activities of Daily Living

Paper Information

• Domain: Human-Computer Interaction, Active Learning, Smart Homes, Natural Language Processing
• Keywords: Human-in-the-loop annotation, active learning, natural language, semantic similarity, human activity recognition (HAR) annotation

Research Background and Problem Statement

• What problems or challenges did the authors identify?

  • Existing human activity recognition (HAR) systems struggle to handle long-term changes in user behavior, physical environment variations, or sensor changes.
  • Supervised learning methods require a large amount of manually labeled data, which is costly and unsuitable for large-scale scenarios.
  • Current mainstream methods for collecting user activity data rely on graphical interfaces, which pose accessibility issues and are inconvenient for real-time activity annotation by users.

• Why is this problem important?

  • Accurate activity recognition is critical for providing intelligent and autonomous services in smart homes and assisted living environments.
  • Combining active learning with user annotations can dynamically improve model performance and adapt to changes.

• Motivation and Related Work

  • Current research on active learning focuses on optimizing query strategies and clustering methods, with less attention to how users interact and how to reduce annotation burdens.
  • Existing voice interaction methods are mostly limited to predefined fixed labels and lack the ability to dynamically process natural language inputs.

Proposed Solution

• What methods or solutions did the authors propose?

  • A dialogue-based user interaction system for active learning in HAR systems.
  • Introduction of a dataset containing natural language descriptions of 33 common activities, using semantic similarity measures for annotation.
  • Adoption of an active learning query mechanism (Query by Committee, QbC) to select data points with high uncertainty for annotation.

• What are the innovative aspects of this solution?

  • Integration of natural language dialogue systems, semantic similarity measures, and active learning strategies to automate and simplify the annotation process.
  • Development of a reusable framework that can be extended to different datasets and models through a simple mapping mechanism.
  • Use of follow-up questioning during the annotation process to reduce ambiguity.

• What are the implementation steps and key techniques used?

  1. Model Construction: Use multiple predictive models (e.g., Random Forest) to form a learning committee, supporting the QbC strategy.
  2. Query Selection: Use Kullback-Leibler divergence to detect maximum disagreement among models and select samples for annotation.
  3. Natural Language Processing:
     • Collect and structure natural language descriptions of common activities.
     • Use tools like SpaCy to evaluate the semantic similarity between user-provided descriptions and predefined activities.
  4. Dialogue Management: Build a systematic dialogue framework that dynamically generates follow-up questions based on user responses to minimize ambiguity.
  5. Model Retraining: Continuously improve the learning model based on user-provided annotations.

Research Findings

• What specific results were achieved?

  • Developed and publicly released a dataset containing natural language descriptions of 33 common activities.
  • Experimental results demonstrated that the system could effectively improve model performance through user annotations, achieving active learning.
  • User studies showed that the dialogue-based natural language annotation method was more user-friendly and had a lower learning curve compared to manual annotation with fixed labels.

• What advantages does this solution have over existing methods?

  • Reduced cognitive and operational burden on users, eliminating the need for continuous manual annotation or reliance on technically skilled personnel.
  • Reduced the number of annotated samples required through active learning while making the system more adaptable to long-term dynamic changes.
  • Provided a highly extensible framework that can be adapted to various HAR datasets and application scenarios.

• What were the experimental or evaluation results?

  • Semantic Similarity Performance: Improved the matching success rate between user inputs and predefined activity descriptions from 73.68% to a theoretical maximum of 89.47% (allowing one follow-up question).
  • Active Learning Experiments: Experiments on the CASAS dataset showed that dialogue-based annotation effectively improved model accuracy. When simulating human annotations, the accuracy of the Random Forest model increased from an initial 63.25% to 74.75%.
  • User Satisfaction Testing: System usability testing (SUS) and NASA Task Load Index (NASA-TLX) indicated that the dialogue-based annotation method outperformed in terms of usability and reducing users' cognitive load.

• Limitations and Future Directions

  • Limitations:

    • The study is currently based on a single dataset (CASAS dataset) and lacks validation across multiple datasets.
    • The experiments simulated users' natural language responses and did not fully test the robustness of the voice processing and dialogue system in real-world scenarios.
    • Participants were primarily young adults, lacking tests targeted at the potential primary user group (e.g., older adults).

  • Future Directions:

    • Expand the natural language dataset to include more languages and cultural contexts.
    • Conduct long-term system testing in real home environments.
    • Explore more intelligent dialogue generation mechanisms by integrating spatial and temporal contextual knowledge to create interaction methods that better align with user habits.

Conclusion

This paper proposes a method for annotating daily activities by combining a dialogue-based natural language interface with active learning mechanisms. The effectiveness and generalizability of this method in HAR systems were successfully demonstrated. In the future, this approach can be further applied to diverse environments and domains, advancing the adoption of human-computer interaction in precise activity recognition and smart homes.