4 ms·

I watched the whole thing and I don't understand how you arrived at him having a growing pessimism for the future of VR? I had the opposite impression. He did

by olau 6y ago

I watched the whole thing and I don't understand how you arrived at him having a growing pessimism for the future of VR? I had the opposite impression. He did seem to care about VR as a general purpose system (alternative to traditional monitors), even if it's not quite there yet, both due to hardware and software limitations. He also seemed proud that they were actually shipping stuff.

dougmwne 6y ago
Yes, I do think he was very proud of the shipping product and saw that there were many more achievable gains. My impression of pessimism came from his skepticism that Moore's law would continue long enough to get them to where they really wanted to be on mobile VR, his struggles with latency probably making cloud rendering a poor experience for most, recognition that they were already at the ceiling of their heat and power budget, his belief that foveated rendering was not going to save the day, his recognition that the lenses were now a limiting factor vs the screen for image quality and that other lens designs would be fragile and suboptimal in a shipping product. I didn't mean to say that I thought he was pessimistic about VR overall, just that he didn't seem to think there were any large gains in hardware quality coming soon and it was down to the software to make full use of what was there.
- jonny_wonny 6y ago
  I mean, I think the last thing he said regarding Moore’s Law was the comment about what that AMD exec said, which was basically that they still have tons of room for future improvements.
  - dougmwne 6y ago
    He said he was skeptical of that exec's claim, just not in a super call them out way. I did not take that section to be particularly optimistic. We are probably near the end of die shrinks anyway, and with them, the thermal advantage they bring.
    - yters 6y ago
      The limit is quantum dots, and that doesn't afford very many more doublings.
  - esyir 6y ago
    Specifically, that was Jim Keller who gave the "we've got this" response to that. I've been grumbling about moore's law for some time, but if Keller says that it's possible, I'll believe him on this.
- lhl 6y ago
  I'd recommend anyone interested in the future of VR displays to take the time to watch Doug Lanman's (FRL) recent presentation (IS&T Electronic Imaging (EI) Symposium: Quality Screen Time: Leveraging Computational Displays for Spatial Computing) to get a taste of what's coming: https://www.youtube.com/watch?v=LQwMAl9bGNY https://www.youtube.com/watch?v=LQwMAl9bGNY - it's great, and one of the more optimistic technology talks I've seen in a while. As far as the limits of mobile computation goes, I think the thing to consider is that Qualcomm is already providing Wi-Fi 6E chips that have VR-class latency (<3ms) - this is bog standard, but of there are of course companies like Amimon that have been offering proprietary ultra-low latency (<1ms) solutions for a while now. We'll see if the PS5/PSVR2 offers a wireless solution, but I don't think it's too much to ask for those looking for cutting edge experience to use untethered, local compute - why do you need to strap an RTX 3080 to your face? There's already more than enough power on mobile to provide the basic I/O for presence and engagement in VR.
  - dougmwne 6y ago
    That is an amazing talk. Thanks for sharing. Lanman also seems to indicate that eyetracking is really hard and is still unsolved to the level it would need to be to drive the halfdome 3 prototype. And yes, that bouncing iris totally did give me the willies. It seems that there may eventually be a varifocal HMD with eyetracked fovated rendering, but I'm doubtful it's coming in Oculus Quest 3 or even 4.
    - lhl 6y ago
      Personally, I'd wouldn't count out electronic varifocal coming in Quest 3 (2022?) as it looks like the Half Dome 3 prototype was already TRL6 w/ the electronic varifocal, and Facebook has been making additional moves like acquiring Lemnis earlier this year, which already has a similar-approach full-stack commercial varifocal system. Varifocal would be a huge win for comfort/long-term usage (eg, if FB continues to push on the remote work/productivity/collaboration use cases). While I'm less sure on foveated rendering (although I'm really impressed by DeepFovea https://research.fb.com/publications/deepfovea-neural-reconstruction-for-foveated-rendering-and-video-compression-using-learned-statistics-of-natural-videos/ https://research.fb.com/publications/deepfovea-neural-recons... I wonder what kind of rendering pipeline will be required), I think basic eye tracking will be coming w/ Quest 3 as well - it's not only needed for DeepFocus, but also their Codec Avatars. The stuff that FB has been showing off the past few years is insanely impressive - I think their mixed reality approach beats the pants out of what MS has been doing. I'm really curious to see what Apple has been whittling away at in secret - I'm doubtful if it can compete, but who knows? Google seems a bit out to lunch on all this.
nvarsj 6y ago
Same here. As an engineer, who is naturally very blunt about work, I just heard another engineer speaking and being honest about all the pros and cons. My overall impression is he is very optimistic about things, and proud of what they've accomplished, and there is a lot they can do to make it even better. So overall positive, and I'm glad Carmack is working on the Oculus and I'm happy to buy one.

by olau 6y ago

dougmwne 6y ago

Yes, I do think he was very proud of the shipping product and saw that there were many more achievable gains. My impression of pessimism came from his skepticism that Moore's law would continue long enough to get them to where they really wanted to be on mobile VR, his struggles with latency probably making cloud rendering a poor experience for most, recognition that they were already at the ceiling of their heat and power budget, his belief that foveated rendering was not going to save the day, his recognition that the lenses were now a limiting factor vs the screen for image quality and that other lens designs would be fragile and suboptimal in a shipping product. I didn't mean to say that I thought he was pessimistic about VR overall, just that he didn't seem to think there were any large gains in hardware quality coming soon and it was down to the software to make full use of what was there.

jonny_wonny 6y ago
I mean, I think the last thing he said regarding Moore’s Law was the comment about what that AMD exec said, which was basically that they still have tons of room for future improvements.
- dougmwne 6y ago
  He said he was skeptical of that exec's claim, just not in a super call them out way. I did not take that section to be particularly optimistic. We are probably near the end of die shrinks anyway, and with them, the thermal advantage they bring.
  - yters 6y ago
    The limit is quantum dots, and that doesn't afford very many more doublings.
- esyir 6y ago
  Specifically, that was Jim Keller who gave the "we've got this" response to that. I've been grumbling about moore's law for some time, but if Keller says that it's possible, I'll believe him on this.
lhl 6y ago
I'd recommend anyone interested in the future of VR displays to take the time to watch Doug Lanman's (FRL) recent presentation (IS&T Electronic Imaging (EI) Symposium: Quality Screen Time: Leveraging Computational Displays for Spatial Computing) to get a taste of what's coming: https://www.youtube.com/watch?v=LQwMAl9bGNY https://www.youtube.com/watch?v=LQwMAl9bGNY - it's great, and one of the more optimistic technology talks I've seen in a while. As far as the limits of mobile computation goes, I think the thing to consider is that Qualcomm is already providing Wi-Fi 6E chips that have VR-class latency (<3ms) - this is bog standard, but of there are of course companies like Amimon that have been offering proprietary ultra-low latency (<1ms) solutions for a while now. We'll see if the PS5/PSVR2 offers a wireless solution, but I don't think it's too much to ask for those looking for cutting edge experience to use untethered, local compute - why do you need to strap an RTX 3080 to your face? There's already more than enough power on mobile to provide the basic I/O for presence and engagement in VR.
- dougmwne 6y ago
  That is an amazing talk. Thanks for sharing. Lanman also seems to indicate that eyetracking is really hard and is still unsolved to the level it would need to be to drive the halfdome 3 prototype. And yes, that bouncing iris totally did give me the willies. It seems that there may eventually be a varifocal HMD with eyetracked fovated rendering, but I'm doubtful it's coming in Oculus Quest 3 or even 4.
  - lhl 6y ago
    Personally, I'd wouldn't count out electronic varifocal coming in Quest 3 (2022?) as it looks like the Half Dome 3 prototype was already TRL6 w/ the electronic varifocal, and Facebook has been making additional moves like acquiring Lemnis earlier this year, which already has a similar-approach full-stack commercial varifocal system. Varifocal would be a huge win for comfort/long-term usage (eg, if FB continues to push on the remote work/productivity/collaboration use cases). While I'm less sure on foveated rendering (although I'm really impressed by DeepFovea https://research.fb.com/publications/deepfovea-neural-reconstruction-for-foveated-rendering-and-video-compression-using-learned-statistics-of-natural-videos/ https://research.fb.com/publications/deepfovea-neural-recons... I wonder what kind of rendering pipeline will be required), I think basic eye tracking will be coming w/ Quest 3 as well - it's not only needed for DeepFocus, but also their Codec Avatars. The stuff that FB has been showing off the past few years is insanely impressive - I think their mixed reality approach beats the pants out of what MS has been doing. I'm really curious to see what Apple has been whittling away at in secret - I'm doubtful if it can compete, but who knows? Google seems a bit out to lunch on all this.

nvarsj 6y ago

Same here. As an engineer, who is naturally very blunt about work, I just heard another engineer speaking and being honest about all the pros and cons. My overall impression is he is very optimistic about things, and proud of what they've accomplished, and there is a lot they can do to make it even better. So overall positive, and I'm glad Carmack is working on the Oculus and I'm happy to buy one.