It works well in the video because in that context we are trying to estimate depth relatively to the camera. In real life, we are trying to estimate depth rela

It works well in the video because in that context we are trying to estimate depth relatively to the camera. In real life, we are trying to estimate depth relative to us. In order to fool our brains we need both a very low latency and a high frame-rate. That was one of the major hurdles to solve for VR as well, leading to John Carmack's famous complaint that he can ping across the Atlantic and back faster than he can send a pixel from his desktop to his screen[0][1]. Anyway, back to the video: basically, when comparing 3D movement relative to the camera our brains seem to be more "forgiving". [0] https://twitter.com/id_aa_carmack/status/193480622533120001 https://twitter.com/id_aa_carmack/status/193480622533120001 [1] https://danluu.com/latency-mitigation/ https://danluu.com/latency-mitigation/