4 ms·

Keep in mind that the 90FPS benchmark is merely required to achieve "presence," which is the phenomenon of the brain accepting the virtual reality as reality. H

Keep in mind that the 90FPS benchmark is merely required to achieve "presence," which is the phenomenon of the brain accepting the virtual reality as reality. Higher framerates are always beneficial (as this also reduces latency). > 420Gbps I assume foveated compression (a pet idea of mine that I have no doubt someone else also has) is probably the only way to realistically deliver the stream to the HMD (assuming that some rendering pipeline that can deliver the uncompressed stream exists). You also have to be concerned about micro-saccades, which may increase the effective resolution of the eye.

dsr_ 6y ago
Right, 420Gb/s is the worst-case number. If you can track the fovea accurately, at 100 Hz you need to keep at least a 9 degree circle at full resolution -- more likely an 18 degree circle -- and even so, there's no mechanical system likely that's going to keep up with that motion, so you need a display screen at full resolution across the entire surface, even if you're only painting full resolution across a small sector each frame. Also also note: there's head motion to track and events to respond to. If you're playing a movie, that's pretty much your best case because you only have to pan and tilt a stream of things you already know about. Playing a game is medium bad, because you can predict the range of possible events. Doing AR mediation is worst, because you can't predict the rest of the universe.
mparramon 6y ago
When you say "foveated compression" do you mean >Apple will liberally use an already-known VR technique that involves using eye-tracking to render objects in the user's periphery at a lower fidelity than what the user is focusing on. ?
- lgats 6y ago
  Interesting recent discussions on Foveated Rendering https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=foveated%20rendering&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- zamalek 6y ago
  No foveated compression is not foveated rendering. It is using the same general idea for a different area of the problem: a higher bitrate (on the wire, or across the wireless link) for in-fovea. Due to latency, you can't render the fovea area only; you need to cover quite a bit more in case the eye moves between submitting the command buffer and the final image being ready. Compression can happen much later in the process, where the latency is much more predictable or even fixed. This is useful because massive bandwidth is easiest along traces, harder along wires and hardest wirelessly.

dsr_ 6y ago

Right, 420Gb/s is the worst-case number. If you can track the fovea accurately, at 100 Hz you need to keep at least a 9 degree circle at full resolution -- more likely an 18 degree circle -- and even so, there's no mechanical system likely that's going to keep up with that motion, so you need a display screen at full resolution across the entire surface, even if you're only painting full resolution across a small sector each frame. Also also note: there's head motion to track and events to respond to. If you're playing a movie, that's pretty much your best case because you only have to pan and tilt a stream of things you already know about. Playing a game is medium bad, because you can predict the range of possible events. Doing AR mediation is worst, because you can't predict the rest of the universe.

mparramon 6y ago

When you say "foveated compression" do you mean >Apple will liberally use an already-known VR technique that involves using eye-tracking to render objects in the user's periphery at a lower fidelity than what the user is focusing on. ?

lgats 6y ago
Interesting recent discussions on Foveated Rendering https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=foveated%20rendering&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
zamalek 6y ago
No foveated compression is not foveated rendering. It is using the same general idea for a different area of the problem: a higher bitrate (on the wire, or across the wireless link) for in-fovea. Due to latency, you can't render the fovea area only; you need to cover quite a bit more in case the eye moves between submitting the command buffer and the final image being ready. Compression can happen much later in the process, where the latency is much more predictable or even fixed. This is useful because massive bandwidth is easiest along traces, harder along wires and hardest wirelessly.