Using Open Data to Automatically Generate Localized Analogies

Document Title

Automatically Generating Localized Analogies Using Open Data

Document Information

• Subject Area: Human-Computer Interaction (HCI), Improving Digital Analogies Using Open Data
• Keywords: analogy, numerical understanding, measurement, crowdsourcing, open data, localization, Wikipedia, user adaptation

Research Background and Problem Statement

• Identified Problems:

  1. Numerical information is ubiquitous in news and statistics, yet many people struggle to understand or identify incorrect figures, even in critical areas such as natural disasters, policy, or health.
  2. Using analogies (i.e., "perspectives") to transform complex numbers into easily comprehensible forms has proven to enhance readers' understanding and memory, but current methods face challenges in scalability, localization, and multicultural adaptation.
  3. Existing analogy generation methods rely on crowdsourcing or domain-specific models, which struggle to reach broad audiences and effectively adapt to different countries and cultures.

• Significance of Research:

  • Accurate understanding of numerical information is crucial for voters, consumers, and policymakers.
  • In the context of globalization, conveying complex statistical information to audiences with diverse cultures and languages is increasingly important.

• Motivation and Related Work:

  • Current methods often rely on time-consuming crowdsourcing, which is limited by the characteristics of user groups from specific countries or languages, resulting in restricted localization effects.
  • This study aims to develop a scalable and international solution by leveraging automation and open data resources (e.g., Wikidata and Wikipedia).

Solution

• Proposed Method:

  1. Construct a reference database containing 39,532 fundamental objects by extracting measurement information such as length, height, area, and mass from Wikidata.
  2. Design a model that uses various open data signals (e.g., search volume, text embeddings, N-gram statistics, and Wiki page traffic) to build proxy features for predicting the suitability of analogies.
  3. Simplify the model design, demonstrating that a model based solely on Wikipedia traffic can deliver efficient performance, thereby reducing computational and modeling costs.
  4. Evaluate the model: test its performance in adapting to new measurement types (e.g., demographics) and different cultural contexts (e.g., the U.S. and France).

• Innovations:

  • Utilizing Wikipedia traffic as a proxy feature circumvents the need for manual intervention and complex models, significantly reducing the difficulty and cost of generating adaptive analogies.
  • The model does not require retraining for new domains or cultures, making it easier to scale.

• Implementation Steps:

  1. Data Collection: Use the SPARQL API of Wikidata to retrieve measurement information and links to multiple languages.
  2. Data Processing: Clean and integrate redundant, ambiguous, and impractical objects to establish a high-quality candidate set.
  3. Model Design: Use signals linking reference objects to users as input features to validate their predictive power for analogy suitability.
  4. User Validation: Collect subjective evaluations of analogies through surveys and use them as benchmarks for assessing model performance.

Research Results

Specific Outcomes

• The model can generate high-quality analogies solely based on Wikipedia page traffic signals, which are generally useful and easy for users to understand.
• The model performs well in untrained domains such as demographics and new measurement types (Spearman correlation coefficient ρ up to 0.83).
• The model also demonstrates strong cultural adaptability, generating different and more acceptable analogies for U.S. and French audiences.

Advantages Over Existing Solutions

• Avoids the high cost and cultural singularity of crowdsourcing by achieving automation and scalability through open data resources.
• Adapts to new measurement types and audiences from different languages/cultures without requiring model retraining.
• Leverages Wikipedia, a cross-cultural and global open-source platform, to enhance global applicability.

Experimental or Evaluation Results

1. Model Performance:
   • Analogy generation for existing measurement types: ρ = 0.77 (overall).
   • Outstanding adaptability in new measurement types (e.g., demographics): ρ = 0.83.
2. Cultural Adaptability:
   • U.S. users preferred culturally matched analogies 80% of the time.
   • In experiments, French users favored localized perspectives 59% of the time.
3. User Estimation Experiments:
   • U.S. users using localized analogies had significantly lower estimation errors for measurement values compared to those using non-localized analogies (error magnitude differences of 4x vs. 10x).

Limitations and Future Directions

• Limitations:

  • Open Data does not include all potentially useful common objects (e.g., beds, microwaves).
  • Wikipedia traffic signals exhibit uneven language coverage and insufficient granularity in cross-cultural contexts.
  • Low-quality entries in knowledge bases may interfere with the model, such as ambiguous entities (e.g., unnamed roads or specific sculptures) that may be irrelevant.

• Future Work:

  1. Integrate crowdsourcing to supplement missing information in Wikidata or Wikipedia.
  2. Provide multiple analogy options instead of a single reference to enhance user adaptability.
  3. Explore fine-tuning within regions of the same country, such as differentiating between U.S. and U.K. cultural adaptations.
  4. Validate the model across more cultural and linguistic contexts to ensure universal applicability and fairness.