Do You See What You Mean? Using Predictive Visualizations to Reduce Optimism in Duration Estimates

Title of the Paper

Do You See What You Mean? Using Predictive Visualizations to Reduce Optimism in Duration Estimates

Paper Information

• Research Area: Human-Computer Interaction, Cognitive Bias, Task Duration Estimation Visualization
• Keywords: Planning Fallacy, Predictive Visualization, Time Estimation, Uncertainty, User Decision-Making, Interactive Feedback, Dot Plot, Decision Optimization

Research Background and Problem

• Problem or Challenge:

  1. Planning fallacy is a common cognitive bias where individuals tend to be overly optimistic about the time required to complete tasks.
  2. Existing methods to reduce optimistic bias, such as task decomposition and listing unforeseen events, have limited effectiveness.
  3. Humans struggle to mentally calculate complex probability distributions or mixed distributions.

• Significance: Inaccurate time estimation impacts daily life, career planning, and can lead to significant economic losses (e.g., the Sydney Opera House construction case).

• Research Motivation:

  1. Support tools, such as predictive visualizations, may reduce cognitive load and improve the accuracy of time estimation.
  2. The effectiveness of visualizing uncertainty has been demonstrated in real-world applications like transportation planning, providing a basis for further exploration and validation.

Proposed Solution

• Methodology:

  1. Combine predictive visualizations (quantile dot plots) with traditional cognitive correction methods (e.g., task decomposition and listing unforeseen events) to improve time estimation.
  2. Simulate estimation values and generate personalized visualizations to enhance users' awareness of uncertainty.

• Innovations:

  1. Apply predictive visualizations to time estimation tasks, explicitly displaying probability distributions using quantile dot plots.
  2. Introduce a second-stage decision-making task to test the role of interactive textual feedback or visualization feedback in optimizing decisions.

• Implementation Steps:

  1. Stage 1 (Time Estimation):
     • Collect participants' time estimates for overall tasks, subtasks, and the probability of unforeseen events.
     • Use Monte Carlo sampling to generate probability distributions for task durations.
     • Present 20/50-point quantile dot plots for participants to adjust their estimates.
  2. Stage 2 (Decision Task):
     • Simulate a train-catching scenario after a shopping task, requiring participants to select an optimal departure time.
     • Provide two types of feedback (slider or visualization curve) showing the probability of waiting time and missing the train.

Research Findings

• Specific Results:

  1. Data visualizations significantly reduced participants' optimistic estimates of task durations while increasing their awareness of uncertainty.
  2. Feedback in the decision task (visualization and interactive text) effectively helped participants optimize their choices, reducing the probability of missing the train with only a slight increase in average waiting time.

• Advantages:

  1. Compared to using traditional methods alone, incorporating predictive visualizations led to greater changes in estimates (average increase of 70% in estimated values, 62% increase in uncertainty range).
  2. Visualizations significantly reduced participants' perception of time estimation bias.

• Experimental or Evaluation Results:

  1. Time Estimation Stage:
     • Initial estimates showed less than 50% coverage of simulated task durations, indicating severe underestimation.
     • After adjustments using predictive visualizations, coverage increased to nearly 75%-100%.
     • No significant difference in effectiveness between 20-point and 50-point quantile dot plots.
  2. Decision Stage:
     • Initial choices showed a significant reduction in the probability of missing the train, with a slight increase in waiting time, demonstrating overall optimization.
     • No significant difference in decision accuracy between the visualization group and the text slider group, but the visualization group had longer interaction times, suggesting additional exploration and learning.

• Limitations and Future Directions:

  1. The study did not directly verify the actual accuracy of estimates due to the lack of real task completion time data.
  2. Potential "demand characteristics" bias in the experiment; future studies could improve by introducing physical tasks or incentive mechanisms.
  3. Design broader user interfaces to address different types of uncertainty (e.g., memory bias).
  4. Investigate whether long-term use of predictive visualizations can cultivate users' ability to make more accurate estimates.