"Merging Results Is No Easy Task": An International Survey Study of Collaborative Data Analysis Practices Among UX Practitioners

Title of the Paper

“Merging Results Is No Easy Task”: An International Survey Study of Collaborative Data Analysis Practices Among UX Practitioners

Paper Information

• Subject Area: User Experience (UX), Data Analysis, Collaboration
• Keywords: User Experience, UX, Usability Testing, Data Analysis, Collaboration, Survey Study

Research Background and Problem Statement

• Problems and Challenges:

  • During the usability testing results analysis phase, UX practitioners often face time constraints, which may lead to overlooked critical issues or misinterpretations.
  • Collaborative analysis is considered to enhance the completeness and reliability of the analysis, but due to limited resources, practitioners have fewer opportunities to collaborate in practice.
  • The process of merging different analysis results is often fraught with difficulties, including differing opinions and a lack of technical tools.

• Significance:

  • Data analysis is a core component of UX practice, directly influencing the quality of design improvement recommendations.
  • Emerging technologies (e.g., deep learning and virtual reality) are driving changes in UX analysis methods, yet there is still insufficient understanding of collaborative analysis practices in UX usability testing.

• Research Motivation and Related Work:

  • Previous studies have primarily focused on individual analysis or small-sample surveys, most of which were conducted years ago and lack adaptation to the rapidly evolving technological environment.
  • This study uses a global survey of UX practitioners to comprehensively reveal the current state, challenges, and improvement needs of collaborative practices.

Solution

• Research Methodology:

  • An online survey method was used to investigate the experiences and collaboration practices of 279 UX practitioners from six continents.
  • The study covered topics such as independent analysis practices, collaboration models, challenges, and ideal feature improvements.

• Innovations:

  • This study is the first to comprehensively quantify collaborative data analysis practices among UX practitioners on an international scale.
  • The survey explores how experience and team size influence collaborative practices and identifies UX practitioners' needs for future tool functionalities.

• Implementation Steps:

  • Survey Design: Included three sections—independent data analysis, collaborative practices and challenges, and tool feature improvements.
  • Data Collection: Respondents were recruited through email lists, LinkedIn groups, and other channels.
  • Data Analysis: A combination of quantitative and qualitative data was analyzed, using statistical tests (e.g., Pearson's Chi-square test) to examine correlations between team size and practices.

Research Findings

• Specific Findings:

  • Most practitioners adopt three collaboration models: analyzing different parts of the data independently before collaborating (“divide and conquer”), group discussions for analysis, and independent analysis of the same data followed by collaboration.
  • Improving the reliability of results is one of the primary goals of collaboration, but in practice, it is often deprioritized in favor of generating a higher number of issues and improvement suggestions.
  • Resource constraints, disagreements, and difficulties in merging results are the main challenges in collaborative analysis.

• Advantages Compared to Existing Solutions:

  • With a large sample size, this study provides a detailed understanding of UX practitioners' actual analysis habits and challenges, significantly surpassing earlier small-scale interview studies.
  • Offers recommendations tailored to the current technological and social environment (e.g., designing integrated collaboration platforms).

• Experimental or Evaluation Results:

  • 66% of respondents found the analysis process time-consuming; 70% felt pressured to complete the analysis within a week.
  • Quantitative data analysis revealed that experienced practitioners are more inclined to use customized analysis formats and prioritize collaboration aimed at improving reliability.

• Limitations and Future Directions:

  • Limitations: The data primarily comes from North America, Asia, and Europe, with limited representation from Latin America, Africa, and Oceania; the COVID-19 pandemic may have influenced feedback on remote collaboration.
  • Future Directions:
    • Explore the impact of different cultures on UX practices.
    • Develop platforms that support efficient collaboration and multidimensional data analysis while integrating design tools.
    • Conduct in-depth qualitative research on collaborators' discussions to uncover specific strategies for resolving disagreements.
    • Further investigate how AI-assisted data analysis tools can improve human collaboration processes.

Conclusion

This study systematically reveals the collaborative models, challenges, and needs of UX practitioners during the usability testing data analysis phase, quantifying key issues faced by the UX community through a large-scale survey. The research provides important design guidance for improving data analysis efficiency and reliability while calling for more cross-regional and cross-cultural studies to address the diversity and depth of user experience practice development.