ReCapture: AR-Guided Time-lapse Photography

Title of the Paper

ReCapture: AR-Guided Time-lapse Photography

Paper Information

• Domain: Application of Augmented Reality (AR) and time-lapse photography technology on mobile devices
• Keywords: Augmented Reality, rephotography, time-lapse video, view synthesis, light field reconstruction

Research Background and Problem Statement

• Identified Issues or Challenges: Recording time-lapse videos typically requires prolonged stability of photographic equipment (e.g., tripods and cameras) fixed in a specific scene, which limits device usage and is often difficult to ensure in public environments without interference. Additionally, multi-view time-lapse data is relatively scarce, and existing solutions for constructing such datasets are costly and inconvenient.
• Significance: Addressing these issues will advance applications such as personal photography, historical documentation, and scientific field studies, while significantly reducing the cost of time-lapse data collection.
• Research Motivation and Related Work:
  • Time-lapse recording usually demands specialized equipment and fixed setups, restricting the range of scenarios for data collection. By leveraging AR technology, handheld time-lapse data collection using ordinary mobile devices becomes feasible.
  • Existing research has paid limited attention to handheld time-lapse data collection and how to ensure accuracy and convenience during frequent repeated captures.

Proposed Solution

• Proposed Solution: The ReCapture system utilizes AR guidance to assist users in capturing time-lapse data with handheld mobile devices. It supports three shooting modes (3D mode, image overlay mode, and light field mode) to enable efficient data collection across diverse scenarios.
  • Innovations:
    • Supports ordinary handheld devices without requiring additional fixed equipment.
    • Introduces three modes to adapt to varying scene conditions, addressing issues such as registration failures caused by image or geometric changes.
    • Enables parallel multi-view time-lapse data collection.
• Implementation Steps and Key Technologies:
  • 3D Mode: Utilizes Apple ARKit for world map initialization and tracking, providing high-precision 3D rotation and translation navigation, suitable for close-range shooting.
  • Overlay Mode: Operates without tracking, relying on static semi-transparent image overlays for long-distance shooting or scenarios where tracking fails.
  • Light Field Mode: Captures dense view samples, suitable for high-precision requirements or registration failure scenarios, and supports post-processing image rendering based on view interpolation.
  • Data visualization includes computer-synthesized images, averaged images, and time-lapse displays based on light fields.

Research Outcomes

• Specific Results:
  • Developed the ReCapture application and validated its capability for time-lapse data collection on handheld devices.
  • Designed a user study to evaluate the usability and efficiency of different modes, with results supporting the rationale behind the three modes.
  • Visualized time-lapse data collected via handheld devices, including multi-view videos and light field-based time-lapse displays.
• Advantages:
  • Compared to traditional fixed equipment, ReCapture significantly reduces collection costs and enhances flexibility.
  • Compared to related work, the system supports frequent repeated captures and offers broader adaptability to various scenarios.
• Experimental or Evaluation Results:
  • Experimental results show that 3D mode achieves higher precision in close-range shooting, while overlay mode performs better in efficiency and robustness for long-distance scenarios.
  • User studies indicate that user interface preferences align with the design intentions, and feedback confirms the system's convenience.
• Limitations and Future Directions:
  • Limitations: Precision in close-range scenarios is affected by handheld device tracking noise, and registration failures in long-distance scenarios still require further optimization.
  • Future Directions: Enhance visual tracking and relocalization technologies on mobile devices to significantly expand the applicability of 3D mode; explore automated interface switching modes and improve user experience.

Additional Notes

This study demonstrates the innovative application of AR technology in the field of time-lapse photography, particularly how stable, high-quality repeated captures can be achieved on handheld mobile devices. The system design offers greater flexibility to individual users and researchers, with potential applications in scientific research, cultural heritage preservation, and artistic creation.