The News Says, the Bot Says: How Immigrants and Locals Differ in Chatbot-Facilitated News Reading

Research Background and Issues

• Issues and Challenges: The authors identify significant differences between immigrants and local residents in accessing news information, including language proficiency, knowledge of social context, and varying information needs. These differences may lead immigrant groups to rely more heavily on technology, such as conversational AI (e.g., chatbots), for news consumption. However, they also face challenges in the depth of news reading and comprehension.
• Significance: In the current trend of reader-centric news dissemination, understanding the news consumption behaviors of different social groups is crucial for designing technology tools that meet their needs, particularly for marginalized groups like immigrants with specific requirements.
• Research Motivation: Existing research on supporting news reading has not sufficiently explored the impact of chatbots on different social groups, especially comparative studies between immigrants and local residents. Against this backdrop, the researchers utilize LLM (large language model)-powered chatbots to analyze the differing needs of these groups and optimize technological solutions.

Solution

• Research Methodology: The authors designed an experiment involving 144 participants divided into three groups—local residents, Chinese immigrants, and Vietnamese immigrants. All participants used Microsoft's Copilot chatbot for news reading and interactive Q&A.
• Innovations:
  • Proposed a taxonomy of question types in news reading, including seven categories such as literal comprehension, analytical thinking, and practical guidance.
  • Developed a "news retelling" task to assess the insights gained by different groups from the news and their dependency on information sources.
  • Combined statistical modeling and content analysis to reveal significant behavioral patterns in news consumption and information-seeking among different user groups.
• Implementation Steps and Techniques:
  • The experiment included four stages: preparation (familiarization with the chatbot), news reading (assisted by the chatbot), questionnaire (evaluating the news experience), and reflection (completing the news retelling task).
  • Data analysis involved categorizing participants' question types, quantifying the frequency of information source usage (news articles, chatbots, or external knowledge), and comparing behavioral patterns across the three groups.

Research Findings

• Specific Findings:
  • Participants' questions were categorized into seven types. Local residents were more inclined to ask analytical questions (e.g., opinion integration, broader implications), while immigrants focused more on literal comprehension (e.g., text summaries, word explanations) and practical advice.
  • In the news retelling task, immigrants were more likely than local residents to rely on chatbot feedback to generate specific action suggestions, while referencing the original news content less frequently.
• Comparison with Existing Research:
  • Unlike studies that rely solely on self-reports or eye-tracking to understand news needs, the authors' approach directly captures participants' information needs and uncovers implicit issues with deeper comprehension.
  • In contrast to previous exploratory studies, this work provides systematic empirical evidence revealing significant differences in how immigrants and local residents interact with and understand news content.
• Experimental or Evaluation Results:
  • Immigrants were active in literal comprehension and practical advice regarding news reports but lacked deeper analytical thinking. Notably, they rarely posed questions about the expansive relationships between news content.
  • Surveys indicated that immigrants perceived the value of news reading to be significantly higher than local residents, although there were no significant differences in perceived workload among the three groups.
• Limitations and Future Directions:
  • Limitations: The sample focused on Chinese and Vietnamese immigrants, excluding broader immigrant groups; the experiment's topic was limited to housing issues, and its applicability to other topics remains to be validated.
  • Future Directions:
    • Explore comparative studies with other immigrant groups or linguistic backgrounds.
    • Investigate the applicability of chatbot support across different news topics (e.g., health, education).
    • Enrich data collection methods to capture more natural news reading behaviors.
    • Extend the research to examine how news media professionals and policymakers can leverage these technologies to optimize news dissemination models.