Towards a Reflection in Creative Experience Questionnaire

Title of the Paper

Towards a Reflection in Creative Experience Questionnaire

Bibliographic Information

• Subject Area: Creativity support tools and reflection in Human-Computer Interaction (HCI)
• Keywords: Creative process, creativity, creativity support tools, evaluation, factor analysis, measurement, psychometrics, reflection, reflective practice

Research Background and Problem

• What problems or challenges did the authors identify?
  Despite the evident importance of reflection in the creative process, it remains underexplored in research on creativity support tools (CST). This may be due to the ambiguous definition of reflection and the difficulty of quantifying and measuring it across different contexts.

• Why is this problem important?
  Reflection is a critical component of the creative process, as it can drive open-ended tasks, influence user experience, and foster innovative design. Effective tools for evaluating creative experiences are essential to advancing research on reflection in the CST domain.

• Research Motivation and Related Work
  Existing reflection scales are typically designed for educational or medical contexts and are not fully applicable to technology-supported creative activities. Designing a lightweight tool to measure the presence and extent of reflective situations could fill a significant gap in the CST field.

Solution

• What methods or solutions did the authors propose?
  The authors developed a lightweight self-assessment questionnaire called the "Reflection in Creative Experience Questionnaire (RiCE)" to distinguish the extent of reflection in creative user experiences. The development process included:

  1. Generating initial questionnaire items through a literature review and existing measurement methods.
  2. Conducting expert reviews and factor analysis on the questionnaire items.
  3. Evaluating the questionnaire's utility through user testing in two specific creative technology tool scenarios.

• What is innovative about this solution?
  RiCE focuses on reflection within creative user experiences, proposing four potentially meaningful dimensions of reflection and preliminarily validating its factor structure through statistical analysis. The questionnaire is designed to be lightweight, allowing it to be used alongside other evaluation tools.

• What are the implementation steps and key techniques used?

  1. Questionnaire Development Phase: Generated 115 items, reduced to 16 through expert selection.
  2. Factor Analysis Phase: Conducted statistical analysis using data from 300 participants to identify four main factors in the questionnaire.
  3. User Testing Phase: Evaluated the questionnaire using two novel technology interfaces, examining its correlation with validated scales such as the Self-Reflection and Insight Scale (SRIS) and the Creativity Support Index (CSI).

Research Outcomes

• What specific outcomes were achieved?
  The authors designed the RiCE questionnaire and identified four dimensions:

  • Reflection on Current Process
  • Reflection on Self
  • Reflection on Past Experience
  • Reflection through Experimentation
    The RiCE questionnaire was shown to effectively distinguish the extent of reflective experiences across different technological interfaces and demonstrated significant correlations with validated scales (SRIS and CSI).

• What are its advantages compared to existing solutions?
  Unlike existing reflection measurement tools, RiCE is specifically designed for creativity support tool contexts. Its lightweight design makes it easier to deploy across various creative technology scenarios and quickly assess reflective experiences.

• What were the experimental or evaluation results?

  • Factor analysis indicated that RiCE has a preliminary reasonable factor structure (total variance explained: 54%).
  • User testing demonstrated that RiCE could differentiate reflective dimensions across two different interfaces (story writing and music creation).
  • RiCE showed a moderate positive correlation with the CSI scale and a weak positive correlation with the SRIS scale.

• Limitations and Future Directions

  1. Limitations:

     • The model fit for confirmatory factor analysis (CFA) was not highly acceptable.
     • Each factor in the scale only contains two items, affecting test-retest reliability.
     • Data was primarily collected from Western users, which may introduce cultural/technological bias.
     • Applicability is limited to individual creative scenarios, with unknown suitability for collaborative work.

  2. Future Directions:

     • Increase the number of items to improve questionnaire design and validate with larger samples.
     • Extend testing to more creative domains (e.g., visual arts, programming design).
     • Explore the applicability of RiCE in team-based collaborative creative activities.
     • Design reverse-coded items to enhance the reliability and validity of the factor structure.
     • Further analyze key features of task interfaces that influence reflection.