CHOIR: A Chatbot-mediated Organizational Memory Leveraging Communication in University Research Labs

Paper Title

CHOIR: A Chatbot-mediated Organizational Memory Leveraging Communication in University Research Labs

Publication Info

• Topic area: AI-mediated organizational memory in research labs
• Keywords: organizational memory, chatbot, knowledge management, research labs, AI-assisted documentation, Slack, privacy-awareness, large language models, conversational AI, collaborative systems

Background and Problem

• Problem / challenge: Knowledge shared in university research labs via chat platforms is often lost in message streams, and existing documentation systems are difficult to maintain and navigate. This leads to inefficiencies in knowledge transfer and recurring questions.
• Significance: Efficient knowledge management in research labs is critical due to frequent member turnover, the need for collaborative updates, and the reliance on regular communication for onboarding and project continuity.
• Motivation and related work: Prior approaches, such as wikis and collaborative documents, face challenges like low participation, outdated content, and high maintenance costs. Conversational platforms and AI systems have shown potential for knowledge capture but lack integration into the full organizational memory lifecycle. This paper addresses these gaps by designing and deploying CHOIR.

Solution

• Proposed approach: CHOIR (Chat-based Helper for Organizational Intelligence Repository), a Slack-integrated chatbot that supports organizational memory through document-grounded Q&A, knowledge sharing, conversational knowledge extraction, and AI-assisted document updates.
• Novelty:
  1. Integration of conversational Q&A with document-grounded responses and references.
  2. AI-assisted workflows for extracting and updating organizational memory from Slack conversations.
  3. Privacy-preserving mechanisms for knowledge sharing and documentation updates.
  4. Empirical evaluation of AI-mediated organizational memory in real-world research labs.
• Procedure and key techniques:
  • CHOIR retrieves answers from lab documentation using retrieval-augmented generation (RAG) and GPT-4.
  • Users can share Q&A exchanges publicly or privately, with options for anonymity.
  • Knowledge from Slack conversations is extracted and proposed as document updates.
  • Lab directors review and approve updates through an AI-assisted workflow integrated with GitHub.

Results

• Concrete findings:
  • 107 questions were asked during a one-month deployment, with 42% answered using existing documentation.
  • 38 documentation updates were made, adding 1,994 words to organizational memory.
  • CHOIR facilitated faster information retrieval and reduced directors’ documentation workload.
• Advantage over baselines:
  • CHOIR enabled real-time knowledge retrieval and collaborative updates directly within Slack, reducing reliance on external tools like Google Docs or wikis.
  • Enhanced visibility into documentation gaps and facilitated contributions from lab members.
• Experiments / evaluation:
  • Conducted a one-month field deployment in four university research labs (n=21 participants).
  • Data collected included system interaction logs and log-informed semi-structured interviews.
  • Usage patterns showed differences based on lab culture, with some labs favoring public Q&A and others preferring private interactions.
• Limitations and future work:
  • Limited generalizability due to small sample size and focus on STEM labs.
  • Short deployment period; longer-term impacts remain untested.
  • Students’ reluctance to contribute to documentation due to psychological barriers.
  • Future work includes expanding CHOIR’s knowledge base to external and cross-lab resources, and addressing privacy-awareness tensions.

Summary

CHOIR is a chatbot-mediated system designed to enhance organizational memory in university research labs by integrating document-grounded Q&A, conversational knowledge extraction, and AI-assisted document updates within Slack. A one-month deployment across four labs demonstrated its ability to improve knowledge retrieval, reduce documentation maintenance burdens, and foster collaborative updates. However, challenges such as privacy-awareness tensions and students’ reluctance to contribute highlight the need for further refinement. CHOIR’s approach offers insights into designing AI-mediated knowledge systems that balance individual privacy with organizational transparency and support contextual knowledge sharing.