6 ms·

Latency, bandwidth (and with that, packet loss). Right now the closest tech to do this would be something like WiFi - but the latency is high, and the typical

Latency, bandwidth (and with that, packet loss). Right now the closest tech to do this would be something like WiFi - but the latency is high, and the typical home's WiFi network bandwidth (practically due to the vagaries of radio) is not sufficient. A few reasons this is important: - latency in VR induces motion sickness and operates on a different scale than we're used to for regular network communications. A 50ms lag in a 2D video game is acceptable, a 50ms lag in VR will cause nausea and vomiting. This is a domain where single-digit milliseconds matter a great deal for user comfort. - resolution and refresh rate is everything - VR's viability in large part hinges on the resolution of the screens, which right now are somewhere between "poor" and "mediocre" - nothing so far has come close to replicating 20/20 vision. Moreover you need those screens to refresh far more quickly than a regular monitor - 60Hz doesn't cut it, 90Hz is basically the minimum. Streaming an uncompressed 8K stream at 90Hz over wireless requires more bandwidth than any existing standard can deliver. It's at the edges of our wired capabilities. Of course you can compress the streams, but that not only degrades image quality (which matters a great deal when the screens are replacing your eyesight) but also injects additional latency into the whole affair. It's theoretically possible - but would require developing a lot of custom tech that doesn't exist yet. It would also likely be highly sensitive to the exact particulars of the physical environment - one user may have a perfect experience while the other experiences so much packet loss/reduction in bandwidth that the whole experience is unusable.

kanetw 5y ago
Yeah, the only way this is remotely feasible without massive latency and compression is a 802.11ay link, if it works when you're moving around. And since it's mmWave you'd need a base station in every room and 100Gbit ethernet to connect to it.
- deleted 5y ago
  [deleted]
jjcm 5y ago
I'd actually argue against this - I've been using my Quest recently to stream steam games over wifi on it, and I've been impressed with the capabilities. Overall the experience is seamless 95% of the time, and I don't notice any major differences between it and my HTC Vive. There are hiccups though - wifi will cut out briefly whenever there's any interference on that frequency, and you'll drop a few frames and quality will degrade for a couple seconds. Overall though I highly expect a stream over wifi 6 to be 100% usable for VR.
- potatolicious 5y ago
  There is some promise, but I'd push back on the implication that this is viable as a mainstream option. Right now through either Oculus (Wireless) Link, or something like Virtual Desktop, you can totally stream content from a PC to the headset, but that comes with a big list of asterisks (easily observable if you look at any support channel for both products): - Most people's WiFi sucks, the room they're in has poor signal, or their router is mostly shite. The experience is awful for them. - Most people's PCs are not connected to their network by ethernet, which seems like a crucial part of getting a good experience. - Even under ideal conditions configured by an enthusiast who groks the tech, frame drops are relatively common. - Streaming is generally not possible at full-resolution due to bandwidth limitations. A wireless-first approach presents an additional barrier to one of VR's biggest stumbling blocks - as screen resolutions increase streaming cannot keep up. So getting the setup to work well right now requires a pretty knowledgeable user. Even assuming we can improve on this, I will wager that "95% effective, visible degradation a few seconds at a time at random intervals" is enough of a problem to be a hard stop on mainstream adoption. This is the hole VR is in generally - the tech is "good enough" for enthusiasts, but punishing to the mainstream.
  - jjcm 5y ago
    I'd agree with all of these caveats. I'll also say that I suspect part of the reason why my experience is good is my PC is hardwired, and my headset is ~2 meters away from my wifi router. I'll also add to what you mention about the initial setup - it's an absolute pain in the ass... every single time. Takes me about 10min to do the setup dance each time I set up air link. In general though I see all these as being fixable. A dedicated transmitter and improvements in the UX of this can address all of these.
basch 5y ago
>Right now the closest tech to do this would be something like WiFi Is that true? 802.11ad (wigig) should be able to handle it. https://www.networkworld.com/article/2172394/understanding-where-802-11ad-wigig-fits-into-the-gigabit-wi-fi-picture.html https://www.networkworld.com/article/2172394/understanding-w... Have you ever used a wigig dock? You plug everything into your dock (hdmi, usb, ethernet) and it all gets beamed to the computer. ~10 microseconds latency. edit: I see another commentor mentioned its successor 802.11ay. Same thing applies, except that 802.11ad is a delivery from the past, not a future promise. we are already there.
- kanetw 5y ago
  Unfortunately still not high enough for low-compressed (DSC 3:1 ~= 9.2Gbit/s) or uncompressed video (~= 27.5Gbit/s) at least at our resolution. But might be viable for lower-res displays.
  - basch 5y ago
    What about splitting up the load? Pre-processing happens on the floor cube, with final work done on the headset. The headset doesnt have to be a video screen only.
    - kanetw 5y ago
      Could be doable, and might be something we investigate in future iterations. I don't think it's worth it for this one, though.
    - freemint 5y ago
      The intermediary stages take more information then the final picture. So by moving part of the processing to the device you increase bandwidth requirements
      - basch 5y ago
        The thought crossed my mind, but here is an example that I am not sure actually ends up being helpful but illustrates a point. Imagine some kind of game with rain where water is collecting on your lenses. You could render and compress the frame, send it over, and distort it further You could even render the first source at 60fps, and the rain at 120fps. Or the first image comes over at a lower resolution, is upscaled, and then the rain effect is rendered at full resolution. The same could apply to synthetic film grain. Compressing a more pristine image and then adding film grain later should allow for significant additional compressibility? Decoupling the rendering into two layers could possibly allow for more resolution and framerate tricks like this? Or even color space upscaling upon display? Would it be possible to send half the color depth in even frames, half in odd, and have a nn up color both frames to their original?
allanrbo 5y ago
How about just a short cable to a box on the desk in front of you? The weight and heat of strapping it all on your head sounds tiring...
- thebigman433 5y ago
  Having to play around a tether the whole time isnt a super great experience. The wire gets in the way and you have to think about it constantly.
- potatolicious 5y ago
  This is pretty much the status quo for tethered VR - you're hooked up to a PC sitting on your desk. It works, but has major usability issues that IMO make it a complete long-term dead-end. It removes any interactivity that require the user to move significantly. More than that, even for just typing/using peripherals the cable is constantly in the way.
- LeifCarrotson 5y ago
  How about a short cable to a box on your belt? Maybe a vest for extra tactile feedback. Or even just putting all the heavy stuff like PCBs, heatsinks, and batteries on the back of the strap to balance the screens and optics on the front. My little Petzl headlamp doesn't mount the tiny 23g battery pack on the forehead due to balance and comfort concerns, I can't imagine the Simula fares any better!
  - kanetw 5y ago
    All the heavy stuff (compute pack) is in the back :) We're considering offering a belt clip option, but I'm not sure it's worth it overall.
jwalton 5y ago
Not too far off why evolution put your eyes right next to your brain, if you think about it.

kanetw 5y ago

Yeah, the only way this is remotely feasible without massive latency and compression is a 802.11ay link, if it works when you're moving around. And since it's mmWave you'd need a base station in every room and 100Gbit ethernet to connect to it.

deleted 5y ago
[deleted]

jjcm 5y ago

I'd actually argue against this - I've been using my Quest recently to stream steam games over wifi on it, and I've been impressed with the capabilities. Overall the experience is seamless 95% of the time, and I don't notice any major differences between it and my HTC Vive. There are hiccups though - wifi will cut out briefly whenever there's any interference on that frequency, and you'll drop a few frames and quality will degrade for a couple seconds. Overall though I highly expect a stream over wifi 6 to be 100% usable for VR.

potatolicious 5y ago
There is some promise, but I'd push back on the implication that this is viable as a mainstream option. Right now through either Oculus (Wireless) Link, or something like Virtual Desktop, you can totally stream content from a PC to the headset, but that comes with a big list of asterisks (easily observable if you look at any support channel for both products): - Most people's WiFi sucks, the room they're in has poor signal, or their router is mostly shite. The experience is awful for them. - Most people's PCs are not connected to their network by ethernet, which seems like a crucial part of getting a good experience. - Even under ideal conditions configured by an enthusiast who groks the tech, frame drops are relatively common. - Streaming is generally not possible at full-resolution due to bandwidth limitations. A wireless-first approach presents an additional barrier to one of VR's biggest stumbling blocks - as screen resolutions increase streaming cannot keep up. So getting the setup to work well right now requires a pretty knowledgeable user. Even assuming we can improve on this, I will wager that "95% effective, visible degradation a few seconds at a time at random intervals" is enough of a problem to be a hard stop on mainstream adoption. This is the hole VR is in generally - the tech is "good enough" for enthusiasts, but punishing to the mainstream.
- jjcm 5y ago
  I'd agree with all of these caveats. I'll also say that I suspect part of the reason why my experience is good is my PC is hardwired, and my headset is ~2 meters away from my wifi router. I'll also add to what you mention about the initial setup - it's an absolute pain in the ass... every single time. Takes me about 10min to do the setup dance each time I set up air link. In general though I see all these as being fixable. A dedicated transmitter and improvements in the UX of this can address all of these.

basch 5y ago

>Right now the closest tech to do this would be something like WiFi Is that true? 802.11ad (wigig) should be able to handle it. https://www.networkworld.com/article/2172394/understanding-where-802-11ad-wigig-fits-into-the-gigabit-wi-fi-picture.html https://www.networkworld.com/article/2172394/understanding-w... Have you ever used a wigig dock? You plug everything into your dock (hdmi, usb, ethernet) and it all gets beamed to the computer. ~10 microseconds latency. edit: I see another commentor mentioned its successor 802.11ay. Same thing applies, except that 802.11ad is a delivery from the past, not a future promise. we are already there.

kanetw 5y ago
Unfortunately still not high enough for low-compressed (DSC 3:1 ~= 9.2Gbit/s) or uncompressed video (~= 27.5Gbit/s) at least at our resolution. But might be viable for lower-res displays.
- basch 5y ago
  What about splitting up the load? Pre-processing happens on the floor cube, with final work done on the headset. The headset doesnt have to be a video screen only.
  - kanetw 5y ago
    Could be doable, and might be something we investigate in future iterations. I don't think it's worth it for this one, though.
  - freemint 5y ago
    The intermediary stages take more information then the final picture. So by moving part of the processing to the device you increase bandwidth requirements
    - basch 5y ago
      The thought crossed my mind, but here is an example that I am not sure actually ends up being helpful but illustrates a point. Imagine some kind of game with rain where water is collecting on your lenses. You could render and compress the frame, send it over, and distort it further You could even render the first source at 60fps, and the rain at 120fps. Or the first image comes over at a lower resolution, is upscaled, and then the rain effect is rendered at full resolution. The same could apply to synthetic film grain. Compressing a more pristine image and then adding film grain later should allow for significant additional compressibility? Decoupling the rendering into two layers could possibly allow for more resolution and framerate tricks like this? Or even color space upscaling upon display? Would it be possible to send half the color depth in even frames, half in odd, and have a nn up color both frames to their original?

allanrbo 5y ago

How about just a short cable to a box on the desk in front of you? The weight and heat of strapping it all on your head sounds tiring...

thebigman433 5y ago
Having to play around a tether the whole time isnt a super great experience. The wire gets in the way and you have to think about it constantly.
potatolicious 5y ago
This is pretty much the status quo for tethered VR - you're hooked up to a PC sitting on your desk. It works, but has major usability issues that IMO make it a complete long-term dead-end. It removes any interactivity that require the user to move significantly. More than that, even for just typing/using peripherals the cable is constantly in the way.
LeifCarrotson 5y ago
How about a short cable to a box on your belt? Maybe a vest for extra tactile feedback. Or even just putting all the heavy stuff like PCBs, heatsinks, and batteries on the back of the strap to balance the screens and optics on the front. My little Petzl headlamp doesn't mount the tiny 23g battery pack on the forehead due to balance and comfort concerns, I can't imagine the Simula fares any better!
- kanetw 5y ago
  All the heavy stuff (compute pack) is in the back :) We're considering offering a belt clip option, but I'm not sure it's worth it overall.

jwalton 5y ago

Not too far off why evolution put your eyes right next to your brain, if you think about it.