Reprojection decouples render rate from display rate

What it is

The application can finish a frame at 70 fps and the panel can still light at 90 Hz. The missing display refreshes take the last finished image and project it again under the newest head pose — asynchronous reprojection, also called time warp / ATW. Render rate and display rate come apart: the display follows the head; the renderer supplies a new image when it can, and a warped old one when it cannot.

It keeps the world updating at display rate for head rotation when the application missed vsync. It does not shorten every stage of the chain.

Why it happens

By the time a frame is drawn the head has left the pose it was submitted at. Reprojection samples pose immediately before scanout and warps the colour buffer by rotation (and, with depth, by translation) into the new viewpoint. A rotational warp is a 2D resample, cheap, and especially effective for nods and yaws: the world should stay put in space, so the picture has to slide the other way. Translation needs depth: regions without it become holes after the warp, usually filled from neighbours, so object edges drag a wrong background.

When application rate falls to half the display, a further step estimates an in-between frame from two old ones (ASW / spacewarp). The estimate is tolerable for rigid motion and fails on transparency, particles, and UI with no depth — ghosts and tears. Reprojection is therefore decoupling, not a free correct new frame: pose is current; occlusion and movers may still be last frame’s.

Turn it off and an application miss becomes a missed refresh. Leave it on and the miss is covered, at the price of artifacts.

Studying it

Hold application frame rate below display rate under control (fixed 45/90, 60/90), compare reprojection on versus off, and rotational warp only versus depth-aware translational warp. Use high-speed video or a photodiode to see whether the display still runs full refresh, and a checklist to count edge holes and hand ghosts.

Independent variables: application-to-display ratio, reprojection kind, whether the scene has transparency and near-hand objects. Dependent variables: actual emission refresh, world-slip error during rotation, artifact counts, ratings of “stuck to the head” versus “mushy edges”.

Keep two questions apart: did the display miss a beat, and is the content right. Asking only “does it feel smooth” will score artifacts as success.

Where it stops holding

An application that always meets display rate barely exercises reprojection; value has to be measured under missed frames. Rotational viewing gains the most; near-hand manipulation can “stick” a hand at the wrong depth when the depth estimate is wrong. If video-see-through camera frames take the same warp, hole-fill is last frame’s real world and the artifact reads as a ghost. Reprojection cannot repair a stall in simulation: physics and animation still tick at application rate; only the camera is warped.

Applying it

• Treat display rate as the hard constraint and application rate as best-effort; leave rotational reprojection on by default so a missed frame does not become a missed refresh.
• Mark near-hand, transparent, and depth-less overlays as “do not invent fill”; punch holes with a conservative colour or redraw that layer, rather than neighbour-smear.
• Do not treat “70 fps average” as done — check that each 90 Hz emission had a fresh pose. That is whether decoupling is in effect.
• How to check: load the scene until the application sits stably below display rate, then turn the head. The world should still stick at display rate; inspect hands and transparent edges frame by frame for smear. Stick plus rubbed edges means decoupling is working and fill policy needs a change — not that reprojection should be turned off.

Related

• Same group: N1.11.1 End-to-end latency is a chain; optimizing only helps the longest segment · N1.11.3 Dropped frames appear as the world jumping during head motion, not slowing · N1.11.4 Latency jitter is harder to adapt to than mean latency
• Nearby: N1.05 Motion-to-Photon Latency · N1.12 Refresh Rate and Persistence
• Search terms: asynchronous reprojection · time warp · ASW