4 ms·

>These are a step beyond “it feels like you made the thing happen”, they are more primitive, like maybe “it feels like you are the thing”. That's a great way o

by _dain_ 4y ago

>These are a step beyond “it feels like you made the thing happen”, they are more primitive, like maybe “it feels like you are the thing”. That's a great way of putting it. I imagine VR developers have to deal with this.

fwlr 4y ago
I vividly recall Carmack describing some of the lengths they had to go to to get VR respond to body movements. IIRC it was crucial to get the scene to move around exactly when your head moves around, or else your body would think you’re poisoned/dying or something. The PC would be polling head position and rendering at like 90fps and even that was too much latency so they had this insane hack where the rendered frame was sent down the wire to the headset, and the headset itself had some onboard processor that would skew and shift the frame according to the last data from the accelerometer literally right before putting the frame on screen. https://developer.oculus.com/documentation/native/android/mobile-timewarp-overview/ https://developer.oculus.com/documentation/native/android/mo...
- Ajedi32 4y ago
  > the headset itself had some onboard processor that would skew and shift the frame according to the last data from the accelerometer literally right before putting the frame on screen FWIW that's only needed when you don't have enough bandwidth to send an uncompressed video stream to the headset and need a decompression step that introduces additional latency (e.g. you're sending the video stream over 6 GHz Wi-Fi or USB instead of DisplayPort). Tethered headsets do use a similar technique in situations where the game can't keep up with rendering at 90fps but that's done on the computer's GPU, not on the headset.
  - fwlr 4y ago
    Yeah, on re-reading I don’t know where I got the idea the headset was doing on-board processing. I recalled that rendering was done using positional data and they had to add a compositor post processing step afterwards using newer position data, but I hallucinated that the need for low latency was so extreme that it had to be done on the headset.
    - Ajedi32 4y ago
      Again, it does happen on the headset, but that's only necessary in situations where there's not enough bandwidth to send an uncompressed video stream. See Oculus Link over USB, for example: https://developer.oculus.com/blog/how-does-oculus-link-work-the-architecture-pipeline-and-aadt-explained/ https://developer.oculus.com/blog/how-does-oculus-link-work-... The diagram there shows Timewarp running on both the PC and the headset side. In headsets like the Rift or Valve Index that send video streams over HDMI or DisplayPort that extra step isn't necessary; everything happens on the PC's GPU.
      - fwlr 4y ago
        Ah, ok, that’s how I ended up with the original mistaken belief. I remembered “Timewarp is necessary” and “Timewarp can happen on the headset” and conflated them into “Timewarp necessarily has to happen on the headset”.

by _dain_ 4y ago

fwlr 4y ago

I vividly recall Carmack describing some of the lengths they had to go to to get VR respond to body movements. IIRC it was crucial to get the scene to move around exactly when your head moves around, or else your body would think you’re poisoned/dying or something. The PC would be polling head position and rendering at like 90fps and even that was too much latency so they had this insane hack where the rendered frame was sent down the wire to the headset, and the headset itself had some onboard processor that would skew and shift the frame according to the last data from the accelerometer literally right before putting the frame on screen. https://developer.oculus.com/documentation/native/android/mobile-timewarp-overview/ https://developer.oculus.com/documentation/native/android/mo...

Ajedi32 4y ago
> the headset itself had some onboard processor that would skew and shift the frame according to the last data from the accelerometer literally right before putting the frame on screen FWIW that's only needed when you don't have enough bandwidth to send an uncompressed video stream to the headset and need a decompression step that introduces additional latency (e.g. you're sending the video stream over 6 GHz Wi-Fi or USB instead of DisplayPort). Tethered headsets do use a similar technique in situations where the game can't keep up with rendering at 90fps but that's done on the computer's GPU, not on the headset.
- fwlr 4y ago
  Yeah, on re-reading I don’t know where I got the idea the headset was doing on-board processing. I recalled that rendering was done using positional data and they had to add a compositor post processing step afterwards using newer position data, but I hallucinated that the need for low latency was so extreme that it had to be done on the headset.
  - Ajedi32 4y ago
    Again, it does happen on the headset, but that's only necessary in situations where there's not enough bandwidth to send an uncompressed video stream. See Oculus Link over USB, for example: https://developer.oculus.com/blog/how-does-oculus-link-work-the-architecture-pipeline-and-aadt-explained/ https://developer.oculus.com/blog/how-does-oculus-link-work-... The diagram there shows Timewarp running on both the PC and the headset side. In headsets like the Rift or Valve Index that send video streams over HDMI or DisplayPort that extra step isn't necessary; everything happens on the PC's GPU.
    - fwlr 4y ago
      Ah, ok, that’s how I ended up with the original mistaken belief. I remembered “Timewarp is necessary” and “Timewarp can happen on the headset” and conflated them into “Timewarp necessarily has to happen on the headset”.