Bitacora: A Toolkit for Supporting NonProfits to Critically Reflect on Social Media Data Use

Title of the Paper

Bitacora: A Toolkit for Supporting Non-Profits to Critically Reflect on Social Media Data Use

Paper Information

• Subject Area: Human-Computer Interaction (HCI), Social Innovation, Non-Profit Organizations, Social Media Data Applications
• Keywords: Social Media Data, User-Generated Content, Non-Governmental Organizations (NGOs), Non-Profit Organizations, Humanitarian Contexts, Data Work, Data Experts, Data Annotation, Real Data, Toolkit, Design

Research Background and Issues

• Problems and Challenges:

  • Non-profit organizations face complexities and challenges in applying social media data.
  • Social media data often lacks contextualization, which can lead to erroneous conclusions or incomplete inferences.
  • Non-profits typically have limited resources, making it difficult to support complex data analyses.
  • Existing toolkit designs may suffer from oversimplification, depoliticization, and insufficient capacity to address complex problems.

• Significance of the Problem:

  • Social media data holds significant potential for addressing humanitarian crises and improving the operational efficiency of non-profits.
  • However, inappropriate use of such data could lead to wasted resources and inaccurate decision-making, with potential life-threatening consequences in humanitarian contexts.

• Research Motivation:

  • To design a toolkit that enables non-profits to independently and critically explore the potential and limitations of social media data.
  • To bridge the gap in technical resources and data analysis skills for non-profits while enhancing their sense of responsibility in data usage.

Solution

• Methodology and Core Solution:

  • Introduced the Bitácora toolkit, which provides a progressive, contextualized framework to help non-profits explore and analyze Twitter data.
  • The toolkit includes a detailed step-by-step manual and two online computational tools, supporting the entire process from data collection to analysis.

• Innovative Features:

  • Emphasizes "contextualized" data analysis, focusing on the social, historical, and cultural background of data sources.
  • Incorporates reflective design by introducing "friction" into decision-making processes, slowing down innovation to encourage organizations to carefully evaluate the applicability of data.

• Implementation Steps and Techniques:

  1. Problem Definition and Initial Dataset Collection:
     • Search for relevant data using keywords and timeframes, iteratively adjusting search strategies.
     • Provide a decision tree to help users determine whether Twitter data is suitable for their specific problem.
     • Offer a worksheet template for recording problem definitions and observations.
  2. Defining Ground Truth:
     • Analyze the initial dataset tweet by tweet to identify which tweets represent the target problem.
     • Use forms designed by the toolkit to document analysis results and reflect on their alignment with the problem goals.
  3. Contextualized Qualitative Analysis:
     • Use Natural Language Processing (NLP) tools to filter data and generate subsets.
     • Conduct qualitative analysis on the subsets, integrating online and offline information sources to form a comprehensive understanding of the problem.
     • Summarize observations qualitatively using an affinity diagram.

Research Outcomes

• Specific Outcomes:

  • The toolkit successfully encouraged participants to critically reflect on social media data, such as its limitations and potential for misrepresentation.
  • The Bitácora toolkit influenced participants to adjust their goals and expectations for data analysis.

• Advantages Over Existing Solutions:

  • By incorporating contextualization and reflective steps, the toolkit offers a more comprehensive approach than traditional technical tools.
  • It inspired users to engage in deeper problem definition and data exploration rather than merely executing technical tasks.

• Experimental or Evaluation Results:

  • The evaluation involved ten practitioners from three types of non-profits, representing diverse technical capabilities and data analysis backgrounds.
  • Key responses and patterns observed include:
    1. During the Problem Definition Stage:
       • Some participants questioned the limitations of fixed keywords and timeframes, suggesting a more iterative approach to exploring keywords and contexts.
    2. During the Ground Truth Definition Stage:
       • Practitioners tended to adjust their analysis goals to accommodate the limitations of Twitter data, emphasizing its role as a starting point rather than a comprehensive solution.
    3. During the Qualitative Analysis Stage:
       • While qualitative analysis helped uncover the complexity of issues, participants found the process time-consuming and results uncertain, suggesting the need for more support from quantitative tools.

• Limitations and Future Directions:

  • The toolkit's operational capacity and data access may be affected by changes in Twitter's platform policies (e.g., data access restrictions, subscription-based models).
  • Stronger technical support for qualitative processes, such as visualization tools (e.g., word clouds, bar charts), is needed, though caution must be exercised regarding potential biases in quantitative analysis.
  • Consider expanding the toolkit to other social media platforms or data sources and exploring ways to reduce non-profits' reliance on a single platform.

Conclusion: The Bitácora toolkit is an innovative tool designed for non-profits, focusing on promoting critical reflection on the applicability of social media data through a progressive and contextualized approach. The study demonstrates its effectiveness in fostering participants' critical thinking about data, though further optimization is needed to enhance its practicality and long-term impact in daily operations.