4 ms·

I believe latency would not be acceptable with current wireless technology.

jacquesm 12y ago
Wireless video can have approximately one millisecond latency today, I'm not sure how bad that would be wrt to things like nausea but it seems pretty short to me.
- ctdonath 12y ago
  Article sez Carmack's first big fight was convincing Samsung that a two frame delay was too long. WAY too long. And it was important enough that he wrote an "obscene workaround" to avoid that "pretty short" latency. I've been casually following Carmack & Occulus for a while; recurring theme is "X ms delay is too long". IIRC (corrections please?), wasn't until latency got down to 20ms that the nausea etc subsided. That doesn't leave much room to both render a high-res complex image and display it by anything more than the shortest transmission time conceivable.
  - omellet 12y ago
    2 frames at 60Hz is about 33ms, which is way more than 1ms latency from a wireless hop.
    - ctdonath 12y ago
      Reconsider: 2 frames at 60Hz was an intolerable 33ms too long, he needed it down to 16ms total. Your optimistic 1ms latency cuts into that maximum refresh time by 7%, time that will (or is already) in high demand by meaningful/viable applications. You're also assuming a highly customized/optimized scenario for the wireless hop, nigh unto a straight analog signal. Digital processing will extend that, likely adding at least one full frame delay - again, something Carmack severely objected to and fought mightily against.
- comex 12y ago
  The Gear VR is currently advertising 20ms motion-to-photon latency [1], so 1ms wouldn't be a big deal - if you can actually make it 1ms. [1] http://www.roadtovr.com/samsung-gear-vr-official-specifications-20ms-latency/ http://www.roadtovr.com/samsung-gear-vr-official-specificati...
  - pyre 12y ago
    > 20ms motion-to-photon latency > so 1ms wouldn't be a big deal Even if it's 1ms, that means that you only have 19ms left for all of the processing / rendering, including to the I/O delay of getting motion data from the input.
    - melling 12y ago
      How much more work can you do on a desktop CPU/GPU in 19ms than on the mobile hardware? If the desktop hardware is 10x faster, wouldn't it still be a win? Also, consider that you might allow for a small window for when things don't go as planned. So, 15ms to do the computations, for example.
      - pyre 12y ago
        This is all assuming a hard 1ms transmission latency. Usually numbers like that are averaged and this seems like a use-case where 1ms would be an upper bound instead.
        jacquesm 12y ago
        That's a really good point. The tech I'm referring to is used in high end (military simulation) gear but it is definitely todays technology and from what I can read it is an upper bound, not an average but that could be marketing, I haven't tested this stuff. But I've been following the 'wearable dual display' setup since a friend of mine built one in 1983 or so (using two very small monitors mounted to the side of a helmet and a bunch of mirrors (2 100%, 1 50%) to put the image in front of the eyes and that's the best latency for wireless that I've seen to date. Given that is a small fraction of the framerate and a latency rather than a throughput limit (so a constant offset rather than a limitation on quality) it imposes a small overhead (at 50 fps that's 20 ms/frame so 5% of the total time budget). That seems a good trade-off versus being tethered, you could for instance make up for that by letting the rest of the pipeline generate an image that is slightly worse to achieve the same framerate in an untethered situation. Of course any latency increase will always come at some price the question is whether or not that is an acceptable price for the improvement you get along another dimension all this assuming that's an upper bound. (And maybe there is higher end gear that I'm not aware of)
- dubcanada 12y ago
  Your eyes are only made for a certain frame rate. Any more and your eyes will ignore it, but your brain will need to piece it back together (ie nausea). Any less and it will seem choppy. It's a very very thin line at which it goes from nausea too choppy/laggy. Not that a few milliseconds of latency really matter, but if you don't care about 2 milliseconds here and another 2 there, they start adding up. It's much better to try your best to have the lowest possible latency.
  - SixSigma 12y ago
    Eyes don't have a framerate. Fighter pilots have been tested and can identify the type of plane in an image with just one frame at 255 fps. Noticing a flash of light can go into the 1000 fps territory. http://www.100fps.com/how_many_frames_can_humans_see.htm http://www.100fps.com/how_many_frames_can_humans_see.htm
    - apl 12y ago
      While it's true that a biological eye doesn't really have a defined "framerate" like video sensors do, the examples you're providing don't really show that. Sensitivity to very short stimuli doesn't prove temporal resolution in the sense that's applied here; what you want to be looking at is flicker fusion instead. As a cheeky example -- still pictures have approximately zero frames per second temporal resolution but would be able to pick up that 2.5ms plane if their sensitivity was sufficiently high... Measuring ERGs (electro-retinogrammes) indicates that our photoreceptors can't really resolve luminance fluctuations above, at the very limit, 70-80Hz. Even Drosophila's high speed cascade doesn't really modulate beyond 200-250Hz. I'm really doubtful about those claims.
      - ghusbands 12y ago
        Indeed, flicker fusion for humans pretty much tops out at 80Hz, but you're underestimating the effects of smooth pursuit eye movement. For example, if you animate a fast-moving object at 80Hz on a 160Hz display, and follow it with your eyes, you will see two copies of the object. There's really no upper limit for this effect.

Wireless video can have approximately one millisecond latency today, I'm not sure how bad that would be wrt to things like nausea but it seems pretty short to me.

ctdonath 12y ago
Article sez Carmack's first big fight was convincing Samsung that a two frame delay was too long. WAY too long. And it was important enough that he wrote an "obscene workaround" to avoid that "pretty short" latency. I've been casually following Carmack & Occulus for a while; recurring theme is "X ms delay is too long". IIRC (corrections please?), wasn't until latency got down to 20ms that the nausea etc subsided. That doesn't leave much room to both render a high-res complex image and display it by anything more than the shortest transmission time conceivable.
- omellet 12y ago
  2 frames at 60Hz is about 33ms, which is way more than 1ms latency from a wireless hop.
  - ctdonath 12y ago
    Reconsider: 2 frames at 60Hz was an intolerable 33ms too long, he needed it down to 16ms total. Your optimistic 1ms latency cuts into that maximum refresh time by 7%, time that will (or is already) in high demand by meaningful/viable applications. You're also assuming a highly customized/optimized scenario for the wireless hop, nigh unto a straight analog signal. Digital processing will extend that, likely adding at least one full frame delay - again, something Carmack severely objected to and fought mightily against.
comex 12y ago
The Gear VR is currently advertising 20ms motion-to-photon latency [1], so 1ms wouldn't be a big deal - if you can actually make it 1ms. [1] http://www.roadtovr.com/samsung-gear-vr-official-specifications-20ms-latency/ http://www.roadtovr.com/samsung-gear-vr-official-specificati...
- pyre 12y ago
  > 20ms motion-to-photon latency > so 1ms wouldn't be a big deal Even if it's 1ms, that means that you only have 19ms left for all of the processing / rendering, including to the I/O delay of getting motion data from the input.
  - melling 12y ago
    How much more work can you do on a desktop CPU/GPU in 19ms than on the mobile hardware? If the desktop hardware is 10x faster, wouldn't it still be a win? Also, consider that you might allow for a small window for when things don't go as planned. So, 15ms to do the computations, for example.
    - pyre 12y ago
      This is all assuming a hard 1ms transmission latency. Usually numbers like that are averaged and this seems like a use-case where 1ms would be an upper bound instead.
      - jacquesm 12y ago
        That's a really good point. The tech I'm referring to is used in high end (military simulation) gear but it is definitely todays technology and from what I can read it is an upper bound, not an average but that could be marketing, I haven't tested this stuff. But I've been following the 'wearable dual display' setup since a friend of mine built one in 1983 or so (using two very small monitors mounted to the side of a helmet and a bunch of mirrors (2 100%, 1 50%) to put the image in front of the eyes and that's the best latency for wireless that I've seen to date. Given that is a small fraction of the framerate and a latency rather than a throughput limit (so a constant offset rather than a limitation on quality) it imposes a small overhead (at 50 fps that's 20 ms/frame so 5% of the total time budget). That seems a good trade-off versus being tethered, you could for instance make up for that by letting the rest of the pipeline generate an image that is slightly worse to achieve the same framerate in an untethered situation. Of course any latency increase will always come at some price the question is whether or not that is an acceptable price for the improvement you get along another dimension all this assuming that's an upper bound. (And maybe there is higher end gear that I'm not aware of)
dubcanada 12y ago
Your eyes are only made for a certain frame rate. Any more and your eyes will ignore it, but your brain will need to piece it back together (ie nausea). Any less and it will seem choppy. It's a very very thin line at which it goes from nausea too choppy/laggy. Not that a few milliseconds of latency really matter, but if you don't care about 2 milliseconds here and another 2 there, they start adding up. It's much better to try your best to have the lowest possible latency.
- SixSigma 12y ago
  Eyes don't have a framerate. Fighter pilots have been tested and can identify the type of plane in an image with just one frame at 255 fps. Noticing a flash of light can go into the 1000 fps territory. http://www.100fps.com/how_many_frames_can_humans_see.htm http://www.100fps.com/how_many_frames_can_humans_see.htm
  - apl 12y ago
    While it's true that a biological eye doesn't really have a defined "framerate" like video sensors do, the examples you're providing don't really show that. Sensitivity to very short stimuli doesn't prove temporal resolution in the sense that's applied here; what you want to be looking at is flicker fusion instead. As a cheeky example -- still pictures have approximately zero frames per second temporal resolution but would be able to pick up that 2.5ms plane if their sensitivity was sufficiently high... Measuring ERGs (electro-retinogrammes) indicates that our photoreceptors can't really resolve luminance fluctuations above, at the very limit, 70-80Hz. Even Drosophila's high speed cascade doesn't really modulate beyond 200-250Hz. I'm really doubtful about those claims.
    - ghusbands 12y ago
      Indeed, flicker fusion for humans pretty much tops out at 80Hz, but you're underestimating the effects of smooth pursuit eye movement. For example, if you animate a fast-moving object at 80Hz on a 160Hz display, and follow it with your eyes, you will see two copies of the object. There's really no upper limit for this effect.