Do We Run How We Say We Run? Formalization and Practice of Governance in OSS Communities

Title of the Paper

Do We Run How We Say We Run? Formalization and Practice of Governance in OSS Communities

Paper Information

• Subject Area: Governance and policy practices in Open Source Software (OSS) communities
• Keywords: Open Source Software, Peer Production, Online Communities, Collective Action, OSS Governance

Research Background and Questions

• What problems or challenges did the authors identify?

  • OSS communities often resist traditional organizational rules, but with the trend towards standardization in the OSS ecosystem, more communities are adopting centralized governance models.
  • While OSS projects benefit from adopting formal governance models supported by nonprofit organizations, they may also face limitations or additional administrative burdens.
  • There may be a gap between the actual governance behaviors exhibited by the community and the official policies defined by foundations.

• Why is this issue important?

  • Understanding how OSS projects operate in decentralized environments while maintaining flexibility in the face of formal rules is crucial for understanding the dynamics of modern online community management.
  • Revealing the interaction between OSS foundation governance and community practices can provide valuable insights for designing more effective governance models.

• Research Motivation and Related Work

  • The authors studied the Apache Software Foundation Incubator (ASFI) project to investigate the relationship between OSS community governance and project outcomes.
  • Using mailing lists as a corpus of community activity, they applied natural language processing (NLP) methods to analyze the correlation between governance behaviors and the internalization of foundation policies.

Solutions

• What methods or solutions did the authors propose?

  • Developed a scalable, semi-supervised language modeling pipeline to parse governance behaviors in OSS project emails.
  • Defined and analyzed community governance behaviors and how they reflect the priorities set by foundation policies.

• What is innovative about this solution?

  • Developed a method to evaluate OSS governance dynamics at scale and in real time, particularly through semantic analysis of governance activities in community emails.
  • Proposed two key metrics for governance behaviors: Governed Activity and Policy Internalization.
  • Quantified the degree to which community practices align with foundation policies using semantic similarity.

• What are the implementation steps and key technologies used?

  1. Governance Behavior Extraction:
     • Used mailing list data and NLP techniques (e.g., semantic role labeling) to extract specific activities from sentences.
  2. Governance Activity Clustering:
     • Applied BERTopic and HDBSCAN for density-based clustering of activities to identify recurring patterns.
     • Assigned topic labels to clusters using the cTF-IDF method.
  3. Policy Internalization Measurement:
     • Used semantic similarity (Cross-Encoder model) to compare community activities with foundation policy texts.
  4. Data Analysis:
     • Explored the relationship between foundation policy coverage on different governance topics and the distribution of community governance activities.
     • Investigated the relationship between the degree of policy internalization and the scope of foundation policies.
     • Conducted regression analysis to assess the relationship between governance behaviors and project success.

Research Findings

• What specific findings were obtained?

  • The authors found no significant correlation between community governance activities and ASFI's policy coverage on different topics, indicating a disconnect between community governance priorities and formal policy guidance.
  • However, the degree of policy internalization was higher in topics with broader policy coverage, suggesting that foundation policies in these areas have a significant influence on community behavior.
  • Certain governance topics (e.g., patch handling and voting schedules) showed a positive correlation between community behaviors and project success.

• What advantages does it have compared to existing solutions?

  • Provides a more comprehensive and automated tool for analyzing OSS community governance behaviors, identifying gaps between actual governance practices and formal policies.
  • Uses semantic modeling to quantify policy internalization, offering a more nuanced understanding compared to traditional case studies or qualitative methods.

• What were the experimental or evaluation results?

  • Community governance activities did not fully reflect the focus areas of foundation policies, but higher policy internalization was associated with community behaviors in certain governance topics that contributed to project success.
  • Analysis of mailing lists revealed that documented formal policies do not fully capture the complete picture of daily community practices.

• Limitations and Future Directions

  • Limitations:

    • User data was primarily derived from public mailing lists, excluding private email content, which may have omitted key discussions.
    • The quality of the semi-supervised learning method depends on point density and embedding dimensions, which may underperform in classifying low-frequency activities.
    • The evaluation criteria for Apache Incubator projects (e.g., graduation and retirement) are widely accepted but may not fully represent the definition of OSS success.

  • Future Directions:

    • Expand the scope of the study to investigate governance and community interactions in other OSS foundations, comparing the Apache model with other models.
    • Conduct interviews and qualitative research to gain deeper insights into how communities interpret and practice policies.
    • Explore governance dynamics in independent OSS communities without foundation support, analyzing governance design in decentralized environments.