AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-Based Deep Neural Networks

Document labeling is a critical step in building various machine learning applications. However, the step can be time-consuming and arduous, requiring a significant amount of human efforts. To support an efficient document labeling environment, we present a system called Attentive Interactive Labeling Assistant (AILA). In its core, AILA uses Interactive Attention Module (IAM), a novel module that visually highlights words in a document that labelers may pay attention to when labeling a document. IAM utilizes attention-based Deep Neural Networks which not only support a prediction of which words to highlight but also enable labelers to indicate words that should be assigned a high attention weight while labeling to improve the future quality of word prediction.We evaluated the labeling efficiency and the accuracy by comparing the conditions with and without IAM in our study. The results showed that participants' labeling efficiency increased significantly under the condition with IAM than the condition without IAM, while the two conditions maintained roughly the same labeling accuracy.