Lights, Camera, Autonomy! Generative Video Fusion for Realistic In Situ Evaluation of Autonomous Drones

Autonomous service robots and drones are increasingly deployed in urban spaces, yet evaluating their designs in situ is constrained by permits, logistics, and the cost of repeated user studies. We contribute SHOWTIME, a generative video-fusion framework that synthesizes realistic, controllable evaluation clips by combining structured prompts, encoding navigation behavior, proxemic policies, and task context with real urban scenes. Across two studies, participants rated SHOWTIME’s generative videos on par with physical recordings for realism, behavioral plausibility, ecological appropriateness, and comfort, and could not reliably distinguish real from generated footage; perceptual differences were driven by environmental structure and crowd density rather than modality. A follow-up experiment showed that generative prototyping elicits robust perceptual contrasts across design variants: communicative signaling (lights, functional cues) increases acceptability and comfort, whereas erratic motion reduces them. The pipeline achieves over a 10× reduction in per-clip time compared to physical trials, enabling rapid, human-in-the-loop iteration on behaviors and UI cues. Together, these results demonstrate human-level perceptual fidelity and practical viability, positioning SHOWTIME as a cost-effective, scalable method for refining autonomous systems and their interfaces in real-world urban contexts.