3 ms·

One important differentiating factor between this and other (consumer) HMD's is the scanning display. The Oculus et al need to render a rectangular matrix of e

One important differentiating factor between this and other (consumer) HMD's is the scanning display. The Oculus et al need to render a rectangular matrix of evenly spaced pixels. If you want a larger field of view or higher resolution this comes with an exponential increase in number of pixels. With a high frequency scanning display and eye tracking you are not locked to a static spatial resolution (see https://en.m.wikipedia.org/wiki/Foveated_imaging https://en.m.wikipedia.org/wiki/Foveated_imaging). There's a whole bunch of other problems when it comes to a render pipeline for something links this, but at least on the physical front it enables super high perceived resolution with less overall points of light to render.

pfranz 10y ago
Foveated rendering only became relevant when Nvidia recently started supporting Multi-Resolution Shading. As new techniques and tools like this get better support we can be a lot more efficient with lower powered hardware. It was interesting to hear Carmack last year talk about re-introducing interlacing as a way to reduce latency. Although, I'm just reinforcing what you're saying =)
- xorxornop 10y ago
  Huh, interlacing is a neat idea to reduce latency, but I'm guessing you'd want a form of intra-frame temporal anti-aliasing, otherwise it might look pretty fugly. Whether or not this would neutralise any latency advantage due to the computational overhead of doing this, I'm not sure...
  - pfranz 10y ago
    Found where Carmack talked about it[1]. For VR, the consensus I've seen seem to push for multi-sample AA > full-screen AA > temporal AA (I feel like I've even seen no AA is better than temporal AA). I'm not quite sure if it's performance, architecture (they really prefer forward renderers instead of deferred renderers for low latency), or aesthetics. In the little bit I've done, when just playing with knobs temporal AA looks better to me. Without AA specular highlights are way too distracting and pop too much. This reference[2] talks about the extra velocity buffer needed for temporal AA and the fact it tends to over-blur which can fuzz out fine details (low resolution is a touchy subject in VR). There's a lot of moving pieces when making decisions. In talking about interlacing, he was ideally talking about not having to create a whole image buffer before sending data to the hardware--just rendering parts (well, scanline) of the image that moved. [1] https://youtu.be/gn8m5d74fk8?t=10m54s https://youtu.be/gn8m5d74fk8?t=10m54s [2] https://www.youtube.com/watch?v=Xk3WUk5T2TQ&feature=youtu.be&t=2525 https://www.youtube.com/watch?v=Xk3WUk5T2TQ&feature=youtu.be...
imh 10y ago
>If you want a larger field of view or higher resolution this comes with an exponential increase in number of pixels. How does that work? Shouldn't it be just linear in FOV? Or quadratic if you are talking about solid angle? This doesn't seem like one of those things that grows exponentially.
Someone 10y ago
From what I read here, I think they are aiming for a system that has 'enough' resolution on the entire field of view. 'Enough' is retina resolution, but only on the fovea, and way less the further away you go from the fovea. That can significantly decrease the amount of work needed to render a scene. Moreover, color-wise, you don't need color except for the center of your field of view (you need cones to see color, and cones can only be found in the fovea). Building such a screen probably isn't that hard; you would need to build new production lines because everything out there is built for rectangular grids of equal-sized pixels, but that probably is 'just' a matter of investing money and time. Writing the software to efficiently draw scenes on such a screen probably is a bit harder; there will be lots of optimizations that need to be discovered (what's the equivalent of Bresenham line drawing, for example?) The biggest challenge will be to keep the high-resolution part of your screen centered on where the retina looks. That requires eye tracking and some way to move your screen. Saccades move at hundreds of degrees per second; you would have to match that. If that's what they are building, and they succeed, I think it will take over the world, even if they initially sell it at $5000 an eye.

pfranz 10y ago

Foveated rendering only became relevant when Nvidia recently started supporting Multi-Resolution Shading. As new techniques and tools like this get better support we can be a lot more efficient with lower powered hardware. It was interesting to hear Carmack last year talk about re-introducing interlacing as a way to reduce latency. Although, I'm just reinforcing what you're saying =)

xorxornop 10y ago
Huh, interlacing is a neat idea to reduce latency, but I'm guessing you'd want a form of intra-frame temporal anti-aliasing, otherwise it might look pretty fugly. Whether or not this would neutralise any latency advantage due to the computational overhead of doing this, I'm not sure...
- pfranz 10y ago
  Found where Carmack talked about it[1]. For VR, the consensus I've seen seem to push for multi-sample AA > full-screen AA > temporal AA (I feel like I've even seen no AA is better than temporal AA). I'm not quite sure if it's performance, architecture (they really prefer forward renderers instead of deferred renderers for low latency), or aesthetics. In the little bit I've done, when just playing with knobs temporal AA looks better to me. Without AA specular highlights are way too distracting and pop too much. This reference[2] talks about the extra velocity buffer needed for temporal AA and the fact it tends to over-blur which can fuzz out fine details (low resolution is a touchy subject in VR). There's a lot of moving pieces when making decisions. In talking about interlacing, he was ideally talking about not having to create a whole image buffer before sending data to the hardware--just rendering parts (well, scanline) of the image that moved. [1] https://youtu.be/gn8m5d74fk8?t=10m54s https://youtu.be/gn8m5d74fk8?t=10m54s [2] https://www.youtube.com/watch?v=Xk3WUk5T2TQ&feature=youtu.be&t=2525 https://www.youtube.com/watch?v=Xk3WUk5T2TQ&feature=youtu.be...

imh 10y ago

>If you want a larger field of view or higher resolution this comes with an exponential increase in number of pixels. How does that work? Shouldn't it be just linear in FOV? Or quadratic if you are talking about solid angle? This doesn't seem like one of those things that grows exponentially.

Someone 10y ago

From what I read here, I think they are aiming for a system that has 'enough' resolution on the entire field of view. 'Enough' is retina resolution, but only on the fovea, and way less the further away you go from the fovea. That can significantly decrease the amount of work needed to render a scene. Moreover, color-wise, you don't need color except for the center of your field of view (you need cones to see color, and cones can only be found in the fovea). Building such a screen probably isn't that hard; you would need to build new production lines because everything out there is built for rectangular grids of equal-sized pixels, but that probably is 'just' a matter of investing money and time. Writing the software to efficiently draw scenes on such a screen probably is a bit harder; there will be lots of optimizations that need to be discovered (what's the equivalent of Bresenham line drawing, for example?) The biggest challenge will be to keep the high-resolution part of your screen centered on where the retina looks. That requires eye tracking and some way to move your screen. Saccades move at hundreds of degrees per second; you would have to match that. If that's what they are building, and they succeed, I think it will take over the world, even if they initially sell it at $5000 an eye.