3 ms·

I'd recommend anyone interested in the future of VR displays to take the time to watch Doug Lanman's (FRL) recent presentation (IS&T Electronic Imaging (EI) Sym

by lhl 6y ago

I'd recommend anyone interested in the future of VR displays to take the time to watch Doug Lanman's (FRL) recent presentation (IS&T Electronic Imaging (EI) Symposium: Quality Screen Time: Leveraging Computational Displays for Spatial Computing) to get a taste of what's coming: https://www.youtube.com/watch?v=LQwMAl9bGNY https://www.youtube.com/watch?v=LQwMAl9bGNY - it's great, and one of the more optimistic technology talks I've seen in a while. As far as the limits of mobile computation goes, I think the thing to consider is that Qualcomm is already providing Wi-Fi 6E chips that have VR-class latency (<3ms) - this is bog standard, but of there are of course companies like Amimon that have been offering proprietary ultra-low latency (<1ms) solutions for a while now. We'll see if the PS5/PSVR2 offers a wireless solution, but I don't think it's too much to ask for those looking for cutting edge experience to use untethered, local compute - why do you need to strap an RTX 3080 to your face? There's already more than enough power on mobile to provide the basic I/O for presence and engagement in VR.

dougmwne 6y ago
That is an amazing talk. Thanks for sharing. Lanman also seems to indicate that eyetracking is really hard and is still unsolved to the level it would need to be to drive the halfdome 3 prototype. And yes, that bouncing iris totally did give me the willies. It seems that there may eventually be a varifocal HMD with eyetracked fovated rendering, but I'm doubtful it's coming in Oculus Quest 3 or even 4.
- lhl 6y ago
  Personally, I'd wouldn't count out electronic varifocal coming in Quest 3 (2022?) as it looks like the Half Dome 3 prototype was already TRL6 w/ the electronic varifocal, and Facebook has been making additional moves like acquiring Lemnis earlier this year, which already has a similar-approach full-stack commercial varifocal system. Varifocal would be a huge win for comfort/long-term usage (eg, if FB continues to push on the remote work/productivity/collaboration use cases). While I'm less sure on foveated rendering (although I'm really impressed by DeepFovea https://research.fb.com/publications/deepfovea-neural-reconstruction-for-foveated-rendering-and-video-compression-using-learned-statistics-of-natural-videos/ https://research.fb.com/publications/deepfovea-neural-recons... I wonder what kind of rendering pipeline will be required), I think basic eye tracking will be coming w/ Quest 3 as well - it's not only needed for DeepFocus, but also their Codec Avatars. The stuff that FB has been showing off the past few years is insanely impressive - I think their mixed reality approach beats the pants out of what MS has been doing. I'm really curious to see what Apple has been whittling away at in secret - I'm doubtful if it can compete, but who knows? Google seems a bit out to lunch on all this.

by lhl 6y ago

dougmwne 6y ago

That is an amazing talk. Thanks for sharing. Lanman also seems to indicate that eyetracking is really hard and is still unsolved to the level it would need to be to drive the halfdome 3 prototype. And yes, that bouncing iris totally did give me the willies. It seems that there may eventually be a varifocal HMD with eyetracked fovated rendering, but I'm doubtful it's coming in Oculus Quest 3 or even 4.

lhl 6y ago
Personally, I'd wouldn't count out electronic varifocal coming in Quest 3 (2022?) as it looks like the Half Dome 3 prototype was already TRL6 w/ the electronic varifocal, and Facebook has been making additional moves like acquiring Lemnis earlier this year, which already has a similar-approach full-stack commercial varifocal system. Varifocal would be a huge win for comfort/long-term usage (eg, if FB continues to push on the remote work/productivity/collaboration use cases). While I'm less sure on foveated rendering (although I'm really impressed by DeepFovea https://research.fb.com/publications/deepfovea-neural-reconstruction-for-foveated-rendering-and-video-compression-using-learned-statistics-of-natural-videos/ https://research.fb.com/publications/deepfovea-neural-recons... I wonder what kind of rendering pipeline will be required), I think basic eye tracking will be coming w/ Quest 3 as well - it's not only needed for DeepFocus, but also their Codec Avatars. The stuff that FB has been showing off the past few years is insanely impressive - I think their mixed reality approach beats the pants out of what MS has been doing. I'm really curious to see what Apple has been whittling away at in secret - I'm doubtful if it can compete, but who knows? Google seems a bit out to lunch on all this.