Galileo: Citizen-led Experimentation Using a Social Computing System

Title of the Paper

Galileo: Citizen-led Experimentation Using a Social Computing System

Bibliographic Information

• Subject Area: Social Computing Systems and Citizen Science Experimentation
• Keywords: Social computing systems, citizen science, crowdsourcing, experimental design, online collaboration, randomized experiments, user-driven research, personal informatics, self-tracking, behavioral science.

Research Background and Issues

• Problems or Challenges Identified by the Authors:

  1. Ordinary people often have questions about health issues or behavioral patterns but lack the expertise and resources to scientifically investigate these questions.
  2. The current field of citizen science focuses more on data collection rather than the design and execution of experiments, leaving the public unable to directly address their own questions.
  3. The technical requirements and background knowledge needed for experimental design make it difficult for non-professionals to participate effectively.
  4. There is a lack of platforms and tools to support ordinary people in participating in scientific research, particularly in designing and running randomized controlled experiments.

• Significance of the Problem:

  1. Expanding the range of participants in scientific experiments can enhance societal scientific literacy and help the public better understand science and data-driven processes.
  2. Individually led experiments have significant potential to fill gaps in scientific research and provide tools for individuals to better understand and improve their quality of life.

• Motivation and Related Work:

  1. Existing citizen science projects, such as bird counting and galaxy classification, have expanded public participation but primarily address problems set by experts rather than those initiated by participants themselves.
  2. Unlike systems such as HabitLab and PatientsLikeMe, Galileo is dedicated to enabling ordinary people to design experiments rather than merely contributing data.
  3. Related literature indicates that while intuitive experiences and strong personal motivation often bring innovative perspectives, the lack of systematic support limits their impact.

Solution

• Proposed Method or Solution: Galileo is a social computing system that supports users in designing, reviewing, and running randomized experiments based on intuitive questions. Its main features include:

  1. Experiment Design Process: Step-by-step guidance and on-demand training to help users transform intuitive questions into structured hypotheses.
  2. Review Process: Embedded prompts to guide reviewers in providing feedback.
  3. Automation Process: Management of tasks such as data collection, randomization, and participant reminders during the experiment.

• Innovative Aspects of the Solution:

  1. Tools designed for non-professional users reduce reliance on expert intervention.
  2. Comprehensive support from design to execution enables participants to test their own hypotheses.
  3. Modular tasks and step-by-step tutorials make complex experimental design intuitive and actionable.

• Implementation Steps and Key Technologies:

  1. Users register on the platform, enter the experiment design module, and complete hypothesis formation, condition setting, and measurement selection.
  2. User-designed experiments must pass a review by at least two reviewers before being executed.
  3. The platform supports experiment execution through automated features such as randomization, task reminders, and data anonymization.
  4. Upon experiment completion, participants and designers can view the results and improve the experiment through discussions.

Research Outcomes

• Specific Results:

  1. Quality of Experiment Designs: Platform testing results show that users can design well-structured experiments closely related to their own lives, with 75% of user experiments meeting high-quality standards across 13 evaluation criteria.
  2. Examples of Experiment Execution: Three communities (Kombucha, Open Humans, and Beer) successfully completed and ran 7-day experiments. Despite diverse participant backgrounds, the platform's format helped users complete the initial experimental cycle.
  3. Review and Feedback: Reviewers provided numerous practical improvement suggestions, enhancing the empirical validity and feasibility of experiment designs.

• Advantages:

  1. Automated processes (e.g., data collection and randomization) reduced operational burdens for experiment designers.
  2. Systematic design support helped users overcome the limitations of non-professional knowledge.
  3. Citizen-led experiments complemented existing scientific research, providing a platform to validate intuitive theories from ordinary individuals.

• Experimental or Evaluation Results:

  1. Platform users from 16 countries designed experiments, with 38% of experiments based on personal experiences and 17% demonstrating potential novelty in scientific research.
  2. Community experiment participation rates were 68%, 63%, and 90%, reflecting differences in topic appeal and dropout rates.

• Limitations and Future Directions:

  1. Most experiment designers currently have high educational backgrounds, limiting the platform's accessibility to a broader user base.
  2. Data indicate that two experiments suffered from insufficient sample sizes, highlighting challenges in participant recruitment for public-led experiments.
  3. Future work should focus on improving the platform's accessibility and reducing barriers to experiment implementation through pre-designed templates and social network recruitment strategies.

Conclusion

This paper demonstrates the potential for ordinary users to design and run experiments through the Galileo system, significantly contributing to broadening scientific participation and unlocking individual scientific potential. Although the current system faces challenges in accessibility and sample adequacy, it provides an important technological pathway for advancing the democratization of science.