4 ms·

This is a great breakdown of the early state of VR productivity. There are still a few key items that are barriers for me personally: 1. Eye and face tracking.

by jjcm 5y ago

This is a great breakdown of the early state of VR productivity. There are still a few key items that are barriers for me personally: 1. Eye and face tracking. The author mentions lip syncing and hand tracking. That goes a long way, but it doesn't recreate the video experience to me. There are a few vtubers out there that are doing some seriously impressive tracking, but it requires tons of special equipment at the moment. Once this gets resolved in a consumer model, I think things will really shine for virtual presence. 2. Lighter weight headsets. For longer sessions, a custom headstrap helps a ton, but even still lighter headsets will go a long ways to helping here. 3. Better lenses and better displays. The lower res means you're really forced to work at huge virtual screen sizes, as the author mentions. While this is the least needed item in this list for basic productivity, the screen door effect definitely creates a barrier some of the time. I highly anticipate that we'll see a sizable portion of the population change over to VR for productivity within ten years. It has huge potential, especially with the remote work environment many are in these days. I would love to be able to feel the presence of people around me again, while still not having a commute or the risks associated with being physically in the same room.

ekianjo 5y ago
> change over to VR for productivity within ten years Curious, but why not AR for productivity rather than VR? Both could work, and AR is less invasive.
- Groxx 5y ago
  AR has a lot further to go on a fundamental technology level to reach "usable without uncomfortably dim surroundings". VR already works. E.g. the very best AR stuff you can buy right now is still moderately-bright glowing pixels on top of whatever you're currently looking towards. So if you have too much contrast in the background, or actual light sources, they show right through and ruin much of the visual clarity. As much as I like AR as a concept, it's much further from "ready". It has all the complexities of VR, plus real-world tracking, plus visual overlay - it'll necessarily trail VR until those latter two are "good enough", and they certainly are not at the moment.
  - kderbyma 5y ago
    if they had some form of e-ink like backing it could block the light on the external side and the. render the emitting light on the eyes side
    - kirrent 5y ago
      How do you block light away from the focal plane? There's been research headsets with incredibly long and convoluted indirect light paths so that an attenuation layer could be inserted at a separate focal plane. There's also out of focus attenuation layers which can be surprisingly effective. Neither seems sufficient for a consumer headset.
  - leereeves 5y ago
    Why don't AR headsets use variable tint sunglass lenses to dim the incoming light?
    - Groxx 5y ago
      There have been some tech demos along these lines - basically an LCD filter layer that turns black where the pixels are lighting up. Last I saw, that opaque layer's pixel density was far too coarse (so it blocked too much or too little - it needs like >10x higher density than the highest density consumer screens out now), and every tech I've seen has had a fundamental issue with focus - we can project AR pixels at comfortable focus ranges with complex enough techniques, but nothing exists to project "darkness", so it's at an entirely different visual and focal distance as the pixels, and it never looks quite right. On top of that, you can't really use it to make semi-opaque pixels - you can only darken, and draw brighter stuff on top. So even if you solve ^ all that, you still have to accurately track the world, figure out what's being occluded, and re-draw that along with the pixel you want to draw. Without blocking vision, so the cameras to do all this can't see exactly the same thing you see. ... so every AR headset just adds a pair of literal sunglasses behind it all to dim the world so things don't look quite as bad, and that's the best we have now. In VR with a couple cameras, you... just draw a semi-opaque pixel on top. And it looks perfect.
- Impossible 5y ago
  VR with pass-through might still win this because it's more practical for work scenarios. Arguably this is still AR, but subtly different than HMDs with transparent displays.
- jjcm 5y ago
  I would argue that the future of AR is VR with passthrough. AR fundamentally has issues with light passing through the display screen. VR passthrough is solvable with current tech. AR that solves the problem of not being able to selectively block out light is a breakthrough or two away.
caslon 5y ago
There are actually a few decent choices for eye tracking now (a company that was just acquired by Bytedance had a pretty good eye-tracking standalone), and there are a few crazy headsets that do all of your listed requirements for 1. and more, but they're all in the $1,000+ range. Lighter-weight headsets is something necessary but also something that is in sort of a weird spot. The Quest 2 is actually a lot heavier than average, because it's standalone. As cool as it is, it's regressive in some interesting ways. For better displays, there's already some pretty impressive displays these days if you don't mind jumping up in price a few notches. Enthusiast-range, for sure, but still consumer models.
- rattray 5y ago
  Which device would you overall recommend for all day productivity use?
  - caslon 5y ago
    Honestly, it depends on what your setup is. I would recommend taking a look at the recent Pimax headsets, the Quest 2, and the new Reverb G2 Omnicept, figuring out which features you can't live without (and which price tags you can), then picking one.
the_af 5y ago
Your last paragraph reminded me of Isaac Asimov's novel "The Naked Sun" and his fictional world of Solaria, where people have grown used to living far from each other and only "meet" in VR, and this has happened long enough that people feel repulsed by other people and feel sick if they believe they are in the actual presence of another person.
- dougmwne 5y ago
  Well that one certainly came to pass, didn't it? The pandemic is the best opportunity FB will ever have to push VR into the mainstream. Zuck has also been playing up that it could replace business travel post pandemic and there have been leaked specs for a Quest Pro that would deliver a very cutting edge productivity experience.
  - wlamartin 5y ago
    Years ago I had a dream that VR companies had released an airborne virus such that the only way for humans to experience outside was to wear VR tech. I woke up thinking, "I need to invest in drone deliveries!"
josephg 5y ago
> The author mentions lip syncing and hand tracking. That goes a long way, but it doesn't recreate the video experience to me. The oculus team put together a seriously jawdropping demo of facial tracking using a few cameras in a headset, combined with a lot of AI. The prototype was from a couple years ago. I can't wait until it makes it into commercial headsets. I wonder what the hold up is? Well worth a look if you haven't seen it: https://www.youtube.com/watch?v=v3XcQtoja_Y https://www.youtube.com/watch?v=v3XcQtoja_Y
qq4 5y ago
> risks associated with being physically in the same room What risks are you anticipating in ten years time?
drited 5y ago
The latest firmware update for the Quest 2 (v33 [1]) brought "Link sharpening" which improves text clarity when connecting to your PC through Air Link (wirelessly) or Link (via USB). I tested before and after the update. Before the resolution wasn't quite there for coding etc., but I could definitely work with the post-update resolution. As an aside, a handy cheap way to get more screens in VR with Oculus Link is to insert a headless HDMI plug into your video card. This also works for creating new screens in VR using Immersed although they claim that the virtual screens (paid feature) work better. [1]: https://support.oculus.com/release-notes/ https://support.oculus.com/release-notes/

by jjcm 5y ago

ekianjo 5y ago

> change over to VR for productivity within ten years Curious, but why not AR for productivity rather than VR? Both could work, and AR is less invasive.

Groxx 5y ago
AR has a lot further to go on a fundamental technology level to reach "usable without uncomfortably dim surroundings". VR already works. E.g. the very best AR stuff you can buy right now is still moderately-bright glowing pixels on top of whatever you're currently looking towards. So if you have too much contrast in the background, or actual light sources, they show right through and ruin much of the visual clarity. As much as I like AR as a concept, it's much further from "ready". It has all the complexities of VR, plus real-world tracking, plus visual overlay - it'll necessarily trail VR until those latter two are "good enough", and they certainly are not at the moment.
- kderbyma 5y ago
  if they had some form of e-ink like backing it could block the light on the external side and the. render the emitting light on the eyes side
  - kirrent 5y ago
    How do you block light away from the focal plane? There's been research headsets with incredibly long and convoluted indirect light paths so that an attenuation layer could be inserted at a separate focal plane. There's also out of focus attenuation layers which can be surprisingly effective. Neither seems sufficient for a consumer headset.
- leereeves 5y ago
  Why don't AR headsets use variable tint sunglass lenses to dim the incoming light?
  - Groxx 5y ago
    There have been some tech demos along these lines - basically an LCD filter layer that turns black where the pixels are lighting up. Last I saw, that opaque layer's pixel density was far too coarse (so it blocked too much or too little - it needs like >10x higher density than the highest density consumer screens out now), and every tech I've seen has had a fundamental issue with focus - we can project AR pixels at comfortable focus ranges with complex enough techniques, but nothing exists to project "darkness", so it's at an entirely different visual and focal distance as the pixels, and it never looks quite right. On top of that, you can't really use it to make semi-opaque pixels - you can only darken, and draw brighter stuff on top. So even if you solve ^ all that, you still have to accurately track the world, figure out what's being occluded, and re-draw that along with the pixel you want to draw. Without blocking vision, so the cameras to do all this can't see exactly the same thing you see. ... so every AR headset just adds a pair of literal sunglasses behind it all to dim the world so things don't look quite as bad, and that's the best we have now. In VR with a couple cameras, you... just draw a semi-opaque pixel on top. And it looks perfect.
Impossible 5y ago
VR with pass-through might still win this because it's more practical for work scenarios. Arguably this is still AR, but subtly different than HMDs with transparent displays.
jjcm 5y ago
I would argue that the future of AR is VR with passthrough. AR fundamentally has issues with light passing through the display screen. VR passthrough is solvable with current tech. AR that solves the problem of not being able to selectively block out light is a breakthrough or two away.

caslon 5y ago

There are actually a few decent choices for eye tracking now (a company that was just acquired by Bytedance had a pretty good eye-tracking standalone), and there are a few crazy headsets that do all of your listed requirements for 1. and more, but they're all in the $1,000+ range. Lighter-weight headsets is something necessary but also something that is in sort of a weird spot. The Quest 2 is actually a lot heavier than average, because it's standalone. As cool as it is, it's regressive in some interesting ways. For better displays, there's already some pretty impressive displays these days if you don't mind jumping up in price a few notches. Enthusiast-range, for sure, but still consumer models.

rattray 5y ago
Which device would you overall recommend for all day productivity use?
- caslon 5y ago
  Honestly, it depends on what your setup is. I would recommend taking a look at the recent Pimax headsets, the Quest 2, and the new Reverb G2 Omnicept, figuring out which features you can't live without (and which price tags you can), then picking one.

the_af 5y ago

Your last paragraph reminded me of Isaac Asimov's novel "The Naked Sun" and his fictional world of Solaria, where people have grown used to living far from each other and only "meet" in VR, and this has happened long enough that people feel repulsed by other people and feel sick if they believe they are in the actual presence of another person.

dougmwne 5y ago
Well that one certainly came to pass, didn't it? The pandemic is the best opportunity FB will ever have to push VR into the mainstream. Zuck has also been playing up that it could replace business travel post pandemic and there have been leaked specs for a Quest Pro that would deliver a very cutting edge productivity experience.
- wlamartin 5y ago
  Years ago I had a dream that VR companies had released an airborne virus such that the only way for humans to experience outside was to wear VR tech. I woke up thinking, "I need to invest in drone deliveries!"

josephg 5y ago

> The author mentions lip syncing and hand tracking. That goes a long way, but it doesn't recreate the video experience to me. The oculus team put together a seriously jawdropping demo of facial tracking using a few cameras in a headset, combined with a lot of AI. The prototype was from a couple years ago. I can't wait until it makes it into commercial headsets. I wonder what the hold up is? Well worth a look if you haven't seen it: https://www.youtube.com/watch?v=v3XcQtoja_Y https://www.youtube.com/watch?v=v3XcQtoja_Y

qq4 5y ago

> risks associated with being physically in the same room What risks are you anticipating in ten years time?

drited 5y ago

The latest firmware update for the Quest 2 (v33 [1]) brought "Link sharpening" which improves text clarity when connecting to your PC through Air Link (wirelessly) or Link (via USB). I tested before and after the update. Before the resolution wasn't quite there for coding etc., but I could definitely work with the post-update resolution. As an aside, a handy cheap way to get more screens in VR with Oculus Link is to insert a headless HDMI plug into your video card. This also works for creating new screens in VR using Immersed although they claim that the virtual screens (paid feature) work better. [1]: https://support.oculus.com/release-notes/ https://support.oculus.com/release-notes/