How to categorize collaboration during a collaborative puzzle-solving task? Validation of collaboration profiles using multimodal data in virtual reality context

In high-stakes collaborative situations, a decline in collaboration quality can lead to adverse events with significant consequences. Analyses performed by Human factor (HF) specialists, while effective in identifying and addressing collaboration issues, are case-specific and most of the time performed a posteriori. To address these limitations, our research focuses on a real-time assessment of collaboration processes using multimodal signals collected and analyzed during the activity. Existing collaboration profiles taxonomies face limitations such as a posteriori profiles detection and the absence of quantitative behavioral indicators that can be measured during the activity. Leveraging Virtual Reality (VR), we have developed a framework for evaluating collaboration in controlled setting, testing the effectiveness of a subset of multimodal signals to detect collaboration profiles. We test our approach in a study including 11 stereotyped collaborative scenarios applied to a VR puzzle-solving task. This study reveals the effectiveness of our approach in distinguishing between non-collaborative and highly collaborative profiles. However, challenges arise in discriminating between closely related collaborative profiles. This paper also proposes some guidelines on how to improve the collaboration profile detection framework and address other collaborative situations.