Shing: A Conversational Agent to Alert Customers of Suspected Online-payment Fraud with Empathetical Communication Skills

Title of the Paper

Shing: A Conversational Agent to Alert Customers of Suspected Online-payment Fraud with Empathetical Communication Skills

Paper Information

• Research Domain: Human-Computer Interaction, Digital Finance, AI Conversational Agents
• Keywords: Conversational Agent, Chatbot, Customer Service, Digital Finance, Trust, Privacy

Research Background and Problem

• Problems or Challenges:

  1. With the widespread adoption of digital payment systems, incidents of online payment fraud have increased rapidly, posing a threat to customer assets. Fraud detection often requires service providers to make phone calls to verify customer information, which demands significant human resources.
  2. Existing voice conversational agents (CAs) often struggle to recover conversations after breakdowns, leading to decreased user satisfaction, reduced trust, and conversation abandonment. Strategies for repairing conversations (e.g., repetition, rephrasing, or explaining issues) need further optimization.

• Significance:
  Researching how to design an intelligent voice conversational system capable of effectively repairing conversation breakdowns and enhancing user experience can help optimize customer service efficiency, protect customer assets, and provide design directions for intelligent interactive systems in other domains.

• Motivation and Related Work:

  1. While previous research has primarily focused on text-based chatbots and user-initiated conversations, there is limited research on repair strategies for agent-initiated conversations in voice interactions.
  2. As user expectations for voice conversational agents increase, it is necessary to explore more human-like conversation repair methods, including emotional perception and empathetic expression.

Solution

• Research Methods and System Design:
  The authors designed a voice conversational agent, "Shing," to detect and alert customers about potential online payment fraud while attempting to repair conversation breakdowns using empathetic communication skills. The agent is capable of:

  1. Actively gathering information to detect payment risks.
  2. Persuading customers to terminate risky transactions.
  3. Providing educational information to enhance customers' risk awareness.

• Core Innovations:

  1. Shing employs emotion-adaptive conversation repair strategies, including active feedback listening, expressing empathy, explaining issues, and rephrasing questions.
  2. It utilizes state-of-the-art natural language processing (NLP) models (e.g., BERT) to detect customer intent and emotions and predict transaction risks.
  3. The system architecture integrates Automatic Speech Recognition (ASR), Spoken Language Understanding (SLU), Dialogue Management (DM), and Natural Language Generation (NLG) technologies to intelligently manage the full conversation process, including breakdown repair.

• Implementation Steps:

  1. Data Collection and Risk Identification: Collect customer transaction information via phone calls to assess transaction risks.
  2. Empathy Expression and Repair Strategies: Trigger corresponding repair strategies to reactivate conversations based on different types of breakdowns (e.g., customer silence, irrelevant responses, or questions).
  3. Model Training and Optimization: Train deep learning models (e.g., BERT and RCNN) using large-scale real conversation data, combined with emotion recognition and risk detection modules to refine dialogue generation.

Research Findings

• Specific Results:

  • Compared to baseline models, "Shing" significantly increased the proportion of customers who terminated risky transactions (29.6% vs. 27.3%).
  • Customers were willing to engage in more dialogue turns with Shing (average 5.69 vs. 5.39 turns) and exhibited a stronger willingness to converse.
  • Shing was able to collect more detailed transaction information from customers, effectively supporting risk assessment.

• Advantages and Comparisons:

  • Compared to traditional CAs that rely solely on apology and question repetition mechanisms, Shing bridges the gap with customers through empathetic expressions and personalized explanations, enhancing interaction experience and behavioral outcomes.
  • Its emotional perception capabilities significantly increased customers' willingness to provide information.

• Experimental Evaluation Results:

  • Shing performed exceptionally well in real-world scenarios (57.3% of 144,795 calls experienced natural breakdowns). Compared to industry-standard CA models, customer trust and willingness to provide information were both improved.
  • Although subjective evaluations of trust and privacy concerns did not show statistically significant differences, Shing demonstrated potential advantages in risk education and behavioral influence.

• Limitations and Future Directions:

  • Limitations:
    1. Data sources were primarily concentrated on Chinese digital finance platforms, which may introduce cultural adaptability biases.
    2. The natural breakdown phenomenon and limited responses to follow-up surveys increased the complexity of data analysis.
    3. The isolated effects of repair strategies could not be tested, and evaluating complex models remains challenging.
  • Future Directions:
    1. Explore the application of Shing in other domains (e.g., surveys or psychological counseling) and investigate the impact of voice features (e.g., tone, speed).
    2. Enhance transparency in obtaining customer informed consent and optimize data privacy protection measures.
    3. Conduct in-depth interviews to study customers' genuine perceptions and feedback regarding CAs.

Conclusion

This study designed a voice conversational agent, Shing, with proactive conversation repair capabilities and validated its effectiveness in real-world digital finance scenarios. The results demonstrate that "Shing" significantly improves customer interaction willingness, information collection, and prevention of online payment fraud, providing empirical evidence and practical design insights for the field of HCI.