Can Online Juries Make Consistent, Repeatable Decisions?

Document Title

Can Online Juries Make Consistent, Repeatable Decisions?

Document Information

• Subject Area: Online Group Decision-Making and Social Computing
• Keywords: Online jury, group decision-making, consistency, social influence, preference aggregation, democratic governance

Research Background and Problem

• Identified Problem or Challenge: Traditional jury decision-making processes may lack consistency across multiple cases, potentially raising concerns about fairness and legitimacy. Additionally, existing research predominantly focuses on face-to-face juries, with limited studies on the consistency of online juries.
• Significance: Online juries are emerging as a participatory governance mechanism on digital platforms, and the consistency of their decisions has profound implications for the fairness, credibility, and governance effectiveness of these platforms.
• Research Motivation and Related Work: Previous literature presents conflicting theoretical and experimental results regarding whether group decision-making is more consistent than individual decision-making. Some studies suggest that group decisions are driven by majority opinions, leading to uniformity, while others argue that group discussions may result in polarization and social path dependency, causing inconsistency. This study aims to address the theoretical gap by exploring the consistency of online jury decisions.

Solution

• Method and Solution: An online experiment was designed to directly measure the consistency of group and individual decisions. The experiment employed pseudonym masking to obscure participants' knowledge of group members' histories, ensuring that the same jury could make decisions on two similar cases independently. Individual decisions served as a control group for comparison with group decisions.
• Innovation: The use of "pseudonym masking" technology to construct the experimental environment allowed participants to remain unaware of community history, enabling direct measurement of decision consistency within the same group under different contexts. This approach overcomes the limitation of traditional face-to-face jury studies, which cannot conduct repeated measurements within the same group.
• Implementation Steps and Key Techniques:
  1. Select debate-intensive, balanced decision-making case pairs from the Reddit community;
  2. Recruit participants for four rounds of case judgments (two rounds of group decisions + two rounds of individual decisions), using pseudonym masking to reorganize group members;
  3. Record group members' decisions, discussion content, and preference changes;
  4. Analyze the consistency levels of individual and group decisions using statistical methods;
  5. Administer follow-up questionnaires to measure participants' predictions and reactions regarding decision consistency.

Research Findings

• Specific Results:
  1. Group and individual decision consistency levels were comparable: group decision consistency was 62.92%, while individual decision consistency was 63.88%.
  2. Group decisions enhanced consistency through social discussion, whereas simple aggregation of individual choices (blind selection) significantly reduced consistency to 46.07%.
  3. Group members were more likely to change their initial opinions during discussions to align with the majority viewpoint.
• Advantages: Group discussions facilitated learning among members, making individual decisions within the group more stable. Group decisions avoided the "tyranny of the majority" problem inherent in blind selection, enhancing the overall credibility of decisions.
• Experimental or Evaluation Results: Group members tended to support majority opinions during discussions, with this tendency manifesting through the expansion of majority factions and social polarization effects. Experimental results revealed significant differences between participants' predictions of consistency and actual outcomes for group and individual decisions.
• Limitations and Future Directions: The study was limited by its short duration (7-minute discussions per round) and the simplicity of case content (sourced from online communities). Future research could extend to diverse cases, including legal issues, and explore the consistency of offline jury decisions. Additionally, further analysis of the relationship between discussion quality and decision consistency warrants attention.

Conclusion

Online juries demonstrate stability in decision consistency, offering a democratic and credible solution for online platform governance. However, the social polarization effects of group discussions may lead to excessive reliance on majority tendencies, necessitating the design of more balanced group composition mechanisms to ensure fair representation of minority viewpoints. This study provides theoretical support and practical guidance for the consistency and stability of both online and offline decision-making models.