On a performance crash or see-through failure, fallback must prioritize environment visibility

What it is

The compositor sticks on the last “solid panel” frame. The headset is still lit; the passthrough process is dead. The world is covered by a frozen picture. Worse than black: people think they can still see. Fail-visible fallback requires that on a performance crash, tracking loss, or passthrough-thread death, the first act is to make the real environment the subject of the picture again — not to keep the virtual layer, not to sit on the last frame.

Power-loss cutting the whole optical path is another failure grade. Here the device is still powered; what broke is compute and compositing.

Why it happens

Video-see-through is a pipeline: camera → composite → display. On a timeout anywhere, default implementations often “keep showing the last frame” or “show an error layer.” If that last frame is a large virtual object, freeze welds a baffle that no longer tracks the head onto the eyes. On tracking loss, world-locked content still pasted at the old pose slides through physical space, sweeping stairs and people. A performance crash (thermal throttle, memory exhaustion) usually drops depth and cut-outs first; virtual-on-real becomes a solid cover — the environment is most hidden exactly when the system is least healthy.

Fallback priority therefore has to be written: camera-direct > lower-resolution passthrough > virtual layer gone > freeze or black last. The virtual layer is disposable; environment pixels are not. Error copy must be small, at the edge, and must not replace passthrough.

Studying it

Fault injection: while walking or operating, kill the composite thread, freeze the main thread, cut depth, simulate thermal throttle. Compare freeze-last-frame, cut-to-black, cut-to-camera-direct, drop-virtual-keep-passthrough. Record time from fault to the environment visible again, collisions in that window, and whether people treat the freeze as still tracking.

Fault types: compositor crash, tracking loss, throttle, camera drop. Fallback policies: direct, drop layer, freeze, black. Scores: visibility-recovery time, collisions, times people keep operating a frozen virtual object.

A seated “crash” button will not show the danger of a freeze while walking.

Where it stops holding

Optical see-through has no compositor to freeze; when the microdisplay dies, real light remains. This mainly constrains enclosed video see-through and systems with solid virtual layers. If the camera is dead too, direct has no source: flip-up, quick-release, or an explicit full-view “you cannot see now.” Do not pretend passthrough remains. Fault fallback in remote collaboration also has to tell the other side “environment first here, content paused,” or they will keep pointing at objects that are gone. A dropped frame or two can reproject; do not cut to direct every time and flicker. Safety-critical sites should rather false-cut to direct and interrupt the task than keep a solid layer under throttle.

Applying it

• Independent watchdogs on compositor and passthrough: on timeout, cut to camera-direct and unload the virtual layer. Last virtual frame is not a fallback.
• On tracking loss, hide all world-locked content. Do not keep pasting at the last pose. A small body-locked status may remain; it must not occupy the centre.
• Under thermal throttle, the first cut is solid layers and effects; camera frame rate is last.
• How to check: kill the composite process while walking through a doorway. Within budget, the picture should become passthrough or direct with no virtual layer, doorframe still visible. Still taking a step behind a panel that no longer follows the head means fallback picked the wrong priority.

Related

• Same group：N5.13.1 An overlay hiding a dynamic hazard in the real environment is the highest-grade risk · N5.13.2 While moving, overlay density should drop to leave attention for the environment · N5.13.4 Stairs, crossings and other safety-critical zones need a rule that clears or fades overlays · N5.13.5 Long-term reliance on overlay cues weakens active scanning of the real environment
• Nearby：N5.07 Overlay Density · N5.12 Overlay Density in Real-world Context · N5.08 Video See-through and Optical See-through
• Search terms：fail-visible fallback · passthrough watchdog · graceful degradation