Designing a Data-Driven Survey System: Leveraging Participants' Online Data to Personalize Surveys

Document Title

Designing a Data-Driven Survey System: Leveraging Participants’ Online Data to Personalize Surveys

Document Information

• Subject Area: Human-Computer Interaction (HCI), Data-Driven Survey Systems, User Data Privacy Management
• Keywords: Data-Driven, Survey Systems, Personalized Questionnaires, User Interface, Online Accounts, HCI

Research Background and Problem

• Problems or Challenges:

  • Popular survey platforms (e.g., Qualtrics, SurveyMonkey) lack the functionality to directly import users' online data to enable highly personalized questionnaires.
  • While customized surveys can improve data quality and response rates, their implementation is often complex and technically demanding.
  • Reliance on self-reported questionnaire data limits research precision and increases participant burden.

• Significance:

  • Personalized questionnaires that integrate users' online behavioral data can significantly enhance research quality, such as reducing social desirability bias or improving recall accuracy.
  • User experience and research quality are particularly critical in surveys involving privacy-sensitive and sensitive topics.

• Research Motivation and Related Work:

  • Many fields of research (e.g., social sciences, sports science, and HCI) require higher-quality, more fine-grained data-driven questionnaires.
  • Existing cases (e.g., studies using Fitbit and Instagram) demonstrate the potential of data-driven questionnaires, but the implementation methods are often ad hoc, resource-intensive, and difficult to replicate.

Solution

• Proposed Method or Solution:

  • Develop an open-source platform called “Data-Driven Surveys” (DDS) that supports importing data from users' online platforms (e.g., Fitbit, Instagram, and GitHub) and integrates seamlessly with popular survey platforms like Qualtrics and SurveyMonkey.
  • Provide three core functionalities:
    1. Data-Based Personalization Logic: Modify survey flow or skip questions.
    2. Template-Based Question Design: Use variable substitution to replace placeholders in question templates.
    3. Custom Variables: Extract specific data from users' historical data by defining rules.

• Innovations:

  • DDS significantly simplifies the process of using respondents' online data in surveys, requiring no programming skills.
  • Offers transparent data usage and privacy protection mechanisms, such as limiting access via OAuth protocols, temporary data storage, and no long-term storage within the DDS platform.
  • Open-source and extensible design allows for the addition of more services (e.g., Spotify) and survey platforms in the future.

• Implementation Steps and Technology:

  1. Researchers create projects in DDS and add required data sources (e.g., Fitbit accounts).
  2. Define built-in and custom variables.
  3. Sync variables to the survey platform, where researchers design personalized questionnaires.
  4. Participants authorize online service access via DDS and complete the questionnaire, with data directly transferred to the survey platform.
  5. Data is securely deleted, and researchers download the complete survey results via DDS.

Research Outcomes

• Specific Outcomes:

  • Achieved seamless integration with Qualtrics and SurveyMonkey survey platforms and three data services: Fitbit, Instagram, and GitHub.
  • Participants could see dynamically generated questions based on their personal data in the survey, enhancing engagement and data accuracy.
  • DDS provided detailed privacy protection measures, including transparency of access and data minimization.

• Advantages Over Existing Solutions:

  • Optimized the previously complex and time-consuming process of generating data-driven questionnaires into a simple, non-technical task.
  • The platform is highly extensible and applicable across various fields.
  • Improved the level of personalization, quality, and reliability of survey data.

• Experimental or Evaluation Results:

  • Collected positive feedback from researchers on the need for data-driven features. Most researchers found the data-based questionnaire logic design and personalized template functions highly useful, though some noted that privacy concerns remain a significant challenge.

• Limitations and Future Directions:

  • Limitations:
    1. Currently, DDS only supports Qualtrics, SurveyMonkey, and three data services, requiring enhanced compatibility for broader applicability.
    2. Data-driven questionnaires may introduce sample bias, as participants willing to share online data may dominate.
    3. User privacy and institutional ethics committee constraints may affect adoption.
  • Future Directions:
    1. Expand data services to include platforms like Spotify or additional social media platforms.
    2. Support Experience Sampling Method (ESM) by integrating real-time data to reduce survey fatigue.
    3. Develop privacy-enhancing technologies to ensure de-identified data-driven questionnaire design.
    4. Conduct user testing to evaluate the usability and user acceptance of DDS.