Optimizing the Use of the Sentence Completion Survey Technique in User Research: A Case Study on the Experience of E-Reading

Document Title

Optimizing the Use of the Sentence Completion Survey Technique in User Research: A Case Study on the Experience of E-Reading

Document Information

• Subject Area: Optimization of user research methods and user experience analysis
• Keywords: User research methods, user experience, online surveys, sentence completion technique, e-reading

Research Background and Issues

• What problems or challenges did the authors identify?

  • The Sentence Completion Technique (SCT) has been widely used in psychology, but its application in user experience (UX) research remains limited and lacks systematic exploration.
  • The UX field increasingly demands cost-effective remote methods capable of collecting rich qualitative data, yet how to optimize and design SCT to gather high-quality data remains underexplored.
  • Current research on SCT is concentrated within a few research teams, with fewer than 20 publications available, insufficient to establish comprehensive methodological guidance.
  • Existing studies on e-reading user experience and user needs lack in-depth analysis, particularly regarding user comparisons and innovative ideas.

• Why is this issue important?

  • User research is a critical component of designing and evaluating user experiences. Better understanding and optimizing SCT can more efficiently uncover user needs and frustrations, providing robust support for UX practices.
  • In the e-reading domain, user dissatisfaction and expectations remain key drivers for design improvements, yet there is a lack of lightweight methods to comprehensively understand user needs at a low cost.

• Research Motivation and Related Work

  • The goal of this study is to improve the application of SCT in user research through optimized design while exploring domain-specific user experiences in e-reading.
  • Core questions include: How do sentence prompts influence the quality and quantity of collected data? Which sentence formats are most effective?

Solutions

• What methods or solutions did the authors propose?

  • This study utilized a real-world case to design and implement an online SCT user survey to understand the user experience of e-reading.
  • It compared the data collection performance of different types of sentences (e.g., general questions, comparative questions, redundant questions, extreme questions).
  • Responses from 1,880 participants were collected, and 14,143 user ideas were statistically analyzed.

• What are the innovative aspects of this solution?

  • This study represents one of the most in-depth explorations of SCT in user research to date, featuring a large sample size and detailed quantitative and qualitative analysis.
  • It introduced a systematic evaluation of the impact of sentence formats, providing actionable recommendations for optimizing online user research methods.
  • It expanded the understanding of user frustrations and innovative expectations in the e-reading domain, offering insights for design improvements.

• What were the implementation steps and key techniques used?

  1. Study Design: Designed 13 sentence completion prompts targeting e-reading experiences and conducted pilot testing to optimize sentence prompts.
  2. Survey Implementation: Used Google Forms as the survey tool, employing conditional branching to ensure customized phrasing based on participants' reading habits.
  3. Data Collection: Distributed the questionnaire via social media and e-reading-related communities; collected 1,880 complete responses within two weeks.
  4. Data Analysis: Conducted qualitative and quantitative analysis using SPSS, including calculating response rates, the number of user ideas, types of answers, and originality scores.

Research Findings

• What specific results were achieved?

  • SCT demonstrated high efficiency in remote data collection, with an average response rate of 91%.
  • Different sentence types significantly influenced the quantity, quality, and originality of user responses. For example:
    • Redundant sentences (e.g., repeated prompts to express frustrations) effectively gathered additional data while uncovering deeper dimensions of user experience.
    • Extreme sentences (e.g., inviting users to describe their ideal e-book) had lower response rates but elicited more original ideas.
  • A total of 14,143 user ideas were collected, including rare and innovative perspectives (4.5%), providing valuable inspiration for design improvements.

• What advantages does it have compared to existing solutions?

  • Compared to methods like interviews, SCT offers significant advantages in cost and time efficiency.
  • Compared to standardized questionnaires, SCT can collect deeper and more personalized user feedback.
  • It provided refined data for improving e-reading experiences, surpassing the dimensional limitations of traditional industry surveys.

• What were the experimental or evaluation results?

  • Quantitative analysis revealed that variations in question design significantly impacted response length, quantity, and quality.
  • Specific sentence types (e.g., comparative questions) were easier to answer and yielded rich insights.
  • Qualitative analysis showed that the data collected was more diverse and innovative compared to previous academic studies and industry surveys.

• Limitations and Future Directions

  • Limitations:
    • The lack of randomization in question design may have caused response order effects.
    • The study was based on a single case, and the generalizability of the results remains to be validated.
    • Data analysis was time-consuming, especially for qualitative coding with large sample sizes.
  • Future Directions:
    • Explore additional sentence design variables, such as changes to prompt placement and more complex branching logic.
    • Compare SCT-collected data with other user research methods (e.g., interviews) to explore trade-offs in data quality and analysis effort.
    • Further investigate the value of sentence repetition, particularly in refining feedback for critical user experience areas.

Recommended Design Practices

• Define research objectives and design multiple sentence prompts around the required survey dimensions; prompts should undergo cognitive interviews to ensure clarity and alignment with objectives.
• Use redundant sentences to delve deeper into key areas of user needs but avoid repetitive designs that may cause user fatigue.
• Introduce extreme questions moderately to inspire innovative ideas, ensuring that these questions are not mandatory.
• For complex, multidimensional user experience research, a balanced sentence design (combining traditional question types with innovative prompts) can better support data completeness and diversity.