21 ms·

Oculus Research to Present Focal Surface Display Discovery at SIGGRAPH

212 points by srinathrajaram 9y ago

mattnewport 9y ago
Article headline is misleading. Eliminating nausea is not the primary benefit of this research and isn't even mentioned in the article. It may help a little with nausea in some cases but it won't eliminate it.
- sctb 9y ago
  Thanks, we've reverted the submitted title “Oculus Research Presents Focal Surface Display. Will Eliminate Nausea in VR” to that of the article.
- srinathrajaram 9y ago
  The intention was not to mislead. The linked paper in the article makes a reference to Vergence Accommodation Conflict.So, in a sense, it is mentioned in the article. Link to Paper: https://scontent.oculuscdn.com/v/t64.5771-25/11162698_1898529713737363_2337972817741479936_n.pdf/FSD_paper_authors_version.pdf?_nc_log=1&oh=df1fb54af254dfb447f873312e1bc7c0&oe=594A2C53 https://scontent.oculuscdn.com/v/t64.5771-25/11162698_189852...
Gravityloss 9y ago
Why are holograms / light field displays not technically possible now? I would assume think we have bright and dense enough displays, and can shape the microlenses.
- ccvannorman 9y ago
  The tech is available, but it's very expensive. Oculus has said that this technology is really a stop-gap that vastly improves focal points for the everyday consumer without breaking the bank. I'm betting this tech will get good mileage over the next 5-15 years as hardware catches up enough to render it unnecessary.
- xj9 9y ago
  https://arxiv.org/ftp/arxiv/papers/1506/1506.06668.pdf https://arxiv.org/ftp/arxiv/papers/1506/1506.06668.pdf
- ryandamm 9y ago
  Because n^4 is nasty. Full light field displays have to draw every ray, not just every location; that's two additional dimensions. Resolutions scale badly, and if you make any of the four dimensions lower resolution, you'll get ugly artifacts. Microlenses aren't the limitation. Pixel density is, both in physical manufacturing side and when refresh / drawing.
  - phkahler 9y ago
    >> Because n^4 is nasty. I keep thinking there will be a significant reduction in complexity there. The intensity of a pixel in a hologram is essentially an integral over all surfaces visible from that point. So imagine a rather complex formula applied for each surface for each pixel. Then imagine holographic bounding boxes that compress complex geometry into a few holograms of what's inside. This would reduce the n^4 back down, but the resolution required for holograms is still very very high. But we could use fancy GPUs to evaluate the integrals. Just hand-waving thinking here...
    - erikpukinskis 9y ago
      You can compress most Earth light fields really well. That might be what you're intuiting. There's a ton of redundancy due to the mostly opaque nature of reality. But compressing a light field and projecting one into an eye are totally different things. Your display needs to be capable of displaying all 3n^4 possible intensities at different times. Depending on the scene and where you're looking, you can get away with showing only a smaller handful of those (some megapixels) but the display still needs to be capable of displaying them all. If your display is just fixed LEDs behind a microlens array then you still need n^4 resolution. Like, a megapixel per pixel. Most of the pixels can be off at any given time, but you'll need them all to display arbitrary scenes.
      - ryandamm 9y ago
        It's actually worse than that, for a near-eye display: unless you know where the person is focusing, you actually don't know where the redundancy is, so you have to draw the entire 4D raster. (If you know their focus distance, you can probably just draw a 2D image on the retina and be done -- you get that 4D->2D projection.) Otherwise, the existence of those additional rays is what allows for focus accommodation: as you focus in different planes, it 'shuffles' the light around, to create sharp edges (where similar light rays line up) or blurry foreground/background (similar light rays strike different portions of the retina -- and if any are missing, there is a 'hole' in the blur disk). Pedantic, perhaps... but these are the stakes.
- mhalle 9y ago
  Mostly because of bandwidth and physics. You have to get information into the display in a form that the display can convert to spatial/angular/wavefront information efficiently and with enough fidelity to be believable. Any data compression method has to be compatible with the underlying display technology so you can bring the signal straight into the modulator. In addition, there's a wire problem: you can only bring so much signal into a light modulator because the signals interface through a surface, not a volume. For these reasons and others, true holographic displays are simply not competitive at this time with stereoscope-like VR systems, which rely on tried-and-true, market-driven incoherent-light raster display technology. People have been asking these kinds of "why not" questions about 3D and building various types of 3D display technology in garages and labs for almost two hundred years. With the exception of the discovery of optical holography, it's surprising how much is fundamentally unchanged.
jimrandomh 9y ago
Nausea in VR is already a solved problem, at least for the case where you have a stationary camera. This is for giving people an extra method of depth perception, but one which isn't strictly necessary.
- istorical 9y ago
  Saying nausea is solved in VR is like a casual observer saying space flight is solved after the first Wright brothers flight. I say this as a VR enthusiast who is optimistic about the industry. You can't say it's solved until an HMD can forcefully move your viewport in-HMD without triggering nausea caused by the real body not moving.
  - Pxtl 9y ago
    To be fair, he did say a stationary camera.
- srinathrajaram 9y ago
  Curious to know why you say it is a 'solved' problem. The paper linked in the article claims to have a better solution.
iblaine 9y ago
VR sickness is primarily caused by latency. ie you move your head, the image takes a few milliseconds to respond and you feel dizzy. But there are other types of VR sickness, like the inability to focus on an object. This research improves your ability to focus on objects at different depths. Your vision is less blurry. So yes, this research does help eliminate nausea in VR. To say otherwise is misleading.
- Sean1708 9y ago
  There's a big difference between "will help reduce nausea" and "will eliminate nausea".
- deleted 9y ago
  [deleted]
- cjslep 9y ago
  > So yes, this research does help eliminate nausea in VR. To say otherwise is misleading. Are you responding to someone? Because this doesn't really make sense to me. Can you clarify?
- istorical 9y ago
  Primarily? I'd say it's primarily caused by a disconnect between what the human vestibular system (your sense of balance and spatial orientation) is telling the brain your body is doing vs the acceleration forces that your body is feeling vs what your eyes are telling you about what the body is doing. Or in other words - differences in movement/locomotion your eyes are seeing in HMD vs what your other senses are experiencing. Which probably would include latency as a subset.
  - Balgair 9y ago
    The term in the medical literature you are looking for is 'Simulator Sickness'. There are 2 main theories on it's occurrence, one of which is the issues with the vestibular system. Wikipedia has a good introduction here: https://en.wikipedia.org/wiki/Simulator_sickness https://en.wikipedia.org/wiki/Simulator_sickness To note, simulator sickness is not that new, we have been grappling with this for ~65 years since the first flight simulators. Despite the massive funding that the DoD has at it's fingertips, we have not found a cure for it OR there has not been a lot of work done to find a cure. However, it seems that more time in the simulator does help, thought it may then hinder actual flight performance. Also, the more experienced pilots had a higher occurrence of it. Again, it's not that well understood.
DaveSapien 9y ago
This is a much better (IMO)approach by Nvidia: http://www.fudzilla.com/news/graphics/39762-nvidia-shows-off-its-light-field-vr-headset-at-vrla-2016 http://www.fudzilla.com/news/graphics/39762-nvidia-shows-off...
- IshKebab 9y ago
  If I'm understanding things, that only has two focal planes. They mention the approach of just having several focal planes in the video.
  - randyrand 9y ago
    They time-multiplex the two planes (switching the displays on and off quickly). Amazingly, the brain interprets it as a 3rd, or 4th, or 5th, plane! This Facebook optic uses the same technique.
    - IshKebab 9y ago
      No that's not accurate. The brain may not be able to distinguish the two focal planes but it's just wrong to say "the brain interprets it as a 3rd, or 4th or 5th plane". And this Facebook technique clearly doesn't use the same technique. Instead of a focal plane they have a focal surface that isn't restricted to a plane. It might not be noticeably better (who knows) but it's definitely not the same technique.
  - DaveSapien 9y ago
    Yes that seems so. Must be for cost reasons, I initially thought the new system was like the one they demoed in 2013 (http://lightfield-forum.com/light-field-camera-prototypes/nvidia-near-eye-light-field-display/ http://lightfield-forum.com/light-field-camera-prototypes/nv...), much more like the Lytro light field camera in approatch. So yes as already said, same idea as Oculus.
6stringmerc 9y ago
Ctrl+F "inner ear" - no results. To me, that's going to be the keystone to a functional VR experience. Until there's some relatively non-intrusive mechanism to fool the body's systems into playing along, I'm sorry, I don't think image resolution or refresh or FPS will solve the problem. They're all very important, sure, but I think the biology of the conundrum is the most challenging short term.
- oopsies49 9y ago
  How do astronauts combat nausea in space? Practice. I personally don't expect a technical solution to this any time soon. We just have to acclimatize ourselves.
  - ethbro 9y ago
    > We just have to acclimatize ourselves. Aww. But I already had my scalpels out for DIY inner ear surgery. I just finished watching the YouTube howto and everything... :(
  - falcolas 9y ago
    > We just have to acclimatize ourselves A lot of discomfort for relatively little (at least thus far) gain.
  - shazow 9y ago
    Not everyone is fit to be an astronaut. Not everyone is able to eliminate nausea through reasonable amounts of acclimatization. The target audience for VR is everyone.
    - Houshalter 9y ago
      Everyone? It's always going to be a niche thing. There's lots of people who can't play regular video games because of motion sickness, but the video game industry doesn't suffer.
      - y4mi 9y ago
ryandamm 9y ago
I think this is overstated (though without looking through the prototype, this is only speculation). During normal, outside-of-headset vision, we focus naturally and quickly on whatever we're looking at. We don't spend time with our eyes consciously defocused on subject matter in our foveal view. So anything that's out of focus will tend to be in our peripheral view. So this is a peripheral technology. I think everyone's still looking for the killer additional tech that will make VR perfect -- but it's not about one magic tech bullet, it's about ecosystems slowly growing, and content getting better. (The headsets are better than people think.)
IshKebab 9y ago
I'm a bit skeptical of how much of a problem this really is. I have never noticed it in VR. Perhaps because: 1. Display resolution is still quite low, so really everything is blurry. 2. You will never be able to notice blurriness where you aren't focused anyway because you aren't looking there! Everything is always blurry in your peripheral vision. 3. Surely eye focus is a feedback system, like in cameras? I mean nobody has problems focusing on TVs because your eyes just magically change focal length until the image is sharp. I am stereoblind so maybe it is a big problem for others.
- miketuritzin 9y ago
  Have you tried to focus on something that is under 1 foot from your eyes in VR? From my experience it is virtually impossible, and I think the cause is related to this problem (not 100% sure though).
  - jobigoud 9y ago
    I think for very close objects you need to have a very good match between your interpupillary distance, the interaxial distance of the HMD lenses and the interaxial distance of the virtual cameras. Also it depends on the HMD screen size, at some point parts of the object's image for each eye falls off screen.
  - saltcured 9y ago
    Try closing one eye to rule out bad alignment of the viewing frustrums to your eyes' actual viewing positions and angles.
- gervase 9y ago
  I might be an outlier, in that I don't experience motion sickness / vestibular mismatch from VR, but I suffer from this focal issue pretty severely. When I'm in VR, I notice it slightly, but it's easy to get used to. However, as soon as I get out, I find it very difficult to focus normally again, usually for about 4x the amount of time I was in. This can make it difficult to do anything that benefits from depth perception, such as driving, walking on uneven terrain, etc. If this technique can improve that side effect, it would make quite a difference for me personally. No one else I've talked to about VR have this same experience, so it might only affect a subset of the population.
russdill 9y ago
Would it be possible to detect the focal distance of the eye and change the entire focal depth of the display to keep it always in focus, similar to automated vision testing devices? It could then perform blurring of out of focus objects as a rendering step.
- deleted 9y ago
  [deleted]
- noio 9y ago
  That wouldn't account for the fact that the lenses in your eyes are physically trying to focus on a different plane because of other cues in the scene. If you look at a close by object in VR (like say the gun you're holding) your eyes will automatically try to accommodate to render nearby objects sharp. By doing so they will actually make the gun blurrier, because it (like everything else) is rendered on a medium-distance plane, while your eyes are cued in to focus on a nearby object. Or did I get that completely wrong?
  - russdill 9y ago
    That's why you'd need to have the rendering engine blur things that should not be in focus. From what I understand on how the medical systems work, you'd shine an IR light into the user's eye. The return from the light would allow you to measure the focal distance of the eye. You'd then adjust the lenses of the headset so that the display is exactly that focal distance away. The focal distance of the eyes would get passed to the renderer. The renderer would do proper focal distance bluring to objects that are not at the correct focal distance. The main limitations would likely be the measuring of eye focal distance in realtime, having lenses respond to it in real time, and the possibility that the always in focus screen door effect would cause issues.
    - russdill 9y ago
      Oh, and accommodation really helps your eyes speed up focusing. Try focusing on things near and far with both eyes, and then only one eye. You can still do it with just one eye, but it goes much faster with 2.
mhalle 9y ago
Having a display that can support ocular accommodation (selective focus by the eye) is an important research development, though it will most likely not change the viewer's experience in a radical way. Practical electronic 3D displays requires bandwidth reduction, both data bandwidth for transmission and optical bandwidth to create practical or lower-cost optical modulators. The goal is to use bandwidth reduction techniques that produce little or no visual artifacts. Some of the techniques used are the same as in 2D (spatial discretization, time multiplexing, compression), while others are unique to 3D (view discretization, limits on view angle, elimination of coherence). Head-mounted displays are basically descendants of stereoscopes, the first 3D displays developed by Wheatstone in 1838. Wheatstone's amazing discovery was that you can throw a huge amount of information about the world away, provide just two images from two viewpoints, project them out to infinity in front of a viewer's two eyes using two lenses/light paths, and a vivid sense of 3D is evoked. That's an incredible amount of information reduction from real life. In the traditional stereoscope, accommodation is thrown away, mostly because its really hard to recreate electro-mechanically, but also because we're generally fine with it. Accommodation isn't effective for distant objects (or for even larger depth ranges as we get older we lose our ability to accommodate), so we likely have neural circuitry to discount imperfect accommodation cues. One of the reasons we turn on bright lights when doing detailed work is to stop our eyes down and increase our depth of field, reducing the need for accommodation. However, there have been perennial debates about the physiological impact of conflicting depth cues involving accommodation, and those debates are more interesting in VR where objects can be (virtually) very close to the viewer and the viewer can dynamically change their physical relationship with virtual objects. Until you have a light modulator that can let you experiment with selectively modulating accommodation within a scene, you can't provide real data on how important accommodation (even approximate accommodation) is for a particular application. Can't wait to see the studies. We did some similar focal plane manipulation in holographic video more than a decade ago, for related reasons (see Fig 7): https://www.researchgate.net/publication/255603167_Reconfigurable_image_projection_holograms https://www.researchgate.net/publication/255603167_Reconfigu...
- AndrewKemendo 9y ago
  I would argue it's an important but not the most important thing for VR/AR given what I've seen from consumer feedback. The most important based on real world talking with consumers is a tossup between FOV and latency.
hackjam600 9y ago
+ University grades changing + Facebook hack + Email interception hack + Email accounts hack + Grade Changes hack + Website crashed hack + Word Press Blogs hack + Retrieval of lost file/documents + Erase criminal records hack + Databases hack + Sales of Dumps cards of all kinds + Untraceable Ip + Bank accounts hack + Individual computers hack + Websites hack + Control devices remotely hack + Burner Numbers hack + Verified Paypal Accounts hack + Any social media account hack + Android & iPhone Hack + server crashed hack + Text message interception hack + Twitters hack + Skype hack + Credit cards hacker + We can drop money into bank accounts. + credit score hack + blank credit card sale + Hack and use Credit Card to shop online + Monitor any phone and email address + Tap into anybody's call and monitor their conversation CONTACT:hackjam600@gmail.com
nsxwolf 9y ago
Weird that I haven't noticed this yet when using VR. I wonder if I'll notice next time after watching this.
spyder 9y ago
There is nice table in the paper which compares the capabilities of the different technologies trying to solve the DOF problem in HDMs. http://i.imgur.com/8rdoeS3.png http://i.imgur.com/8rdoeS3.png
AndrewKemendo 9y ago
Honestly I found the paper critically lacking where they attempted to make reference or comparisons to virtual retinal displays. Saying that a VRD is functionally restricted to moderate FOV in comparison to the 120 degree FOV of the Rift - using only the embodiment of the deform-able membrane mirror as reference is ridiculous on it's face. Even a rough version of the deformable mirror AR VRD described by researchers at UNC Chapel Hill [1] accomplishes 100 degrees FOV with accommodation. They went further with the Pinlight achieving 110 in 2014 [2] The technical limit according to our own work for VRD FOV is H: 200o, V: 140o (Combined). So either they're ignoring work in the field intentionally cause they don't want to do VRD or they don't know about it. My guess would be the former. [1]http://telepresence.web.unc.edu/research/dynamic-focus-augmented-reality-display/ http://telepresence.web.unc.edu/research/dynamic-focus-augme... [2]http://www.cs.unc.edu/%7Emaimone/media/pinlights_siggraph_2014.pdf http://www.cs.unc.edu/%7Emaimone/media/pinlights_siggraph_20... edit: I find this whole thing extremely frustrating. Facebook could throw 2 billion dollars at VRD tech and actually get to a working stable consumer grade system if they wanted to - everything is there for it. Why aren't they?
- deleted 9y ago
  [deleted]
- tjoff 9y ago
  Because the requirements to do it well makes it costly, which in turn makes it a gimmick.
  - AndrewKemendo 9y ago
    Why would it be a gimmick if it's costly? Or are you saying the proposed thing is gimmicky?
    - tjoff 9y ago
      It won't get traction if it is costly, and without traction noone will bother developing more than demos and proof of concepts for it. And thus for the mainstream it won't have any real value. Niche markets is another thing but not lucrative enough for a big investment.
cobbzilla 9y ago
i guess whatever the faults of the paper, i like that they do have a product on the market and are demoing and publishing research. magic leap? not so much.
eutropia 9y ago
Wont this also need gaze-tracking to be successful? In their video they described a manually moved camera. Is this technology compatible with foveal rendering?
- drcode 9y ago
  No, I think this technique presents differently-focused areas to the eye all at once (sort of like natural light, just at extremely coarse resolution) and eye tracking is not needed.
migueloller 9y ago
"It may even let people who wear corrective lenses comfortably use VR without their glasses." If just for this, it's a move in the right direction.
- whatnotests 9y ago
  Yes! Many people cannot use the VR Headsets unless they wear contacts, and even then it feels quite unnatural.
highd 9y ago
Sort of strange to describe it as a "discovery" - I'm sure a team of engineers with a variety of fields of expertise spent 1-3 years solving problems that led up to this. A "discovery" would seem to describe something that existed in the aether prior to their work - it seems to diminish the innovation and effort they put in.
sitkack 9y ago
No one has mentioned magic leap.

Article headline is misleading. Eliminating nausea is not the primary benefit of this research and isn't even mentioned in the article. It may help a little with nausea in some cases but it won't eliminate it.

sctb 9y ago
Thanks, we've reverted the submitted title “Oculus Research Presents Focal Surface Display. Will Eliminate Nausea in VR” to that of the article.
srinathrajaram 9y ago
The intention was not to mislead. The linked paper in the article makes a reference to Vergence Accommodation Conflict.So, in a sense, it is mentioned in the article. Link to Paper: https://scontent.oculuscdn.com/v/t64.5771-25/11162698_1898529713737363_2337972817741479936_n.pdf/FSD_paper_authors_version.pdf?_nc_log=1&oh=df1fb54af254dfb447f873312e1bc7c0&oe=594A2C53 https://scontent.oculuscdn.com/v/t64.5771-25/11162698_189852...

Gravityloss 9y ago

Why are holograms / light field displays not technically possible now? I would assume think we have bright and dense enough displays, and can shape the microlenses.

ccvannorman 9y ago
The tech is available, but it's very expensive. Oculus has said that this technology is really a stop-gap that vastly improves focal points for the everyday consumer without breaking the bank. I'm betting this tech will get good mileage over the next 5-15 years as hardware catches up enough to render it unnecessary.
xj9 9y ago
https://arxiv.org/ftp/arxiv/papers/1506/1506.06668.pdf https://arxiv.org/ftp/arxiv/papers/1506/1506.06668.pdf
ryandamm 9y ago
Because n^4 is nasty. Full light field displays have to draw every ray, not just every location; that's two additional dimensions. Resolutions scale badly, and if you make any of the four dimensions lower resolution, you'll get ugly artifacts. Microlenses aren't the limitation. Pixel density is, both in physical manufacturing side and when refresh / drawing.
- phkahler 9y ago
  >> Because n^4 is nasty. I keep thinking there will be a significant reduction in complexity there. The intensity of a pixel in a hologram is essentially an integral over all surfaces visible from that point. So imagine a rather complex formula applied for each surface for each pixel. Then imagine holographic bounding boxes that compress complex geometry into a few holograms of what's inside. This would reduce the n^4 back down, but the resolution required for holograms is still very very high. But we could use fancy GPUs to evaluate the integrals. Just hand-waving thinking here...
  - erikpukinskis 9y ago
    You can compress most Earth light fields really well. That might be what you're intuiting. There's a ton of redundancy due to the mostly opaque nature of reality. But compressing a light field and projecting one into an eye are totally different things. Your display needs to be capable of displaying all 3n^4 possible intensities at different times. Depending on the scene and where you're looking, you can get away with showing only a smaller handful of those (some megapixels) but the display still needs to be capable of displaying them all. If your display is just fixed LEDs behind a microlens array then you still need n^4 resolution. Like, a megapixel per pixel. Most of the pixels can be off at any given time, but you'll need them all to display arbitrary scenes.
    - ryandamm 9y ago
      It's actually worse than that, for a near-eye display: unless you know where the person is focusing, you actually don't know where the redundancy is, so you have to draw the entire 4D raster. (If you know their focus distance, you can probably just draw a 2D image on the retina and be done -- you get that 4D->2D projection.) Otherwise, the existence of those additional rays is what allows for focus accommodation: as you focus in different planes, it 'shuffles' the light around, to create sharp edges (where similar light rays line up) or blurry foreground/background (similar light rays strike different portions of the retina -- and if any are missing, there is a 'hole' in the blur disk). Pedantic, perhaps... but these are the stakes.
mhalle 9y ago
Mostly because of bandwidth and physics. You have to get information into the display in a form that the display can convert to spatial/angular/wavefront information efficiently and with enough fidelity to be believable. Any data compression method has to be compatible with the underlying display technology so you can bring the signal straight into the modulator. In addition, there's a wire problem: you can only bring so much signal into a light modulator because the signals interface through a surface, not a volume. For these reasons and others, true holographic displays are simply not competitive at this time with stereoscope-like VR systems, which rely on tried-and-true, market-driven incoherent-light raster display technology. People have been asking these kinds of "why not" questions about 3D and building various types of 3D display technology in garages and labs for almost two hundred years. With the exception of the discovery of optical holography, it's surprising how much is fundamentally unchanged.

jimrandomh 9y ago

Nausea in VR is already a solved problem, at least for the case where you have a stationary camera. This is for giving people an extra method of depth perception, but one which isn't strictly necessary.

istorical 9y ago
Saying nausea is solved in VR is like a casual observer saying space flight is solved after the first Wright brothers flight. I say this as a VR enthusiast who is optimistic about the industry. You can't say it's solved until an HMD can forcefully move your viewport in-HMD without triggering nausea caused by the real body not moving.
- Pxtl 9y ago
  To be fair, he did say a stationary camera.
srinathrajaram 9y ago
Curious to know why you say it is a 'solved' problem. The paper linked in the article claims to have a better solution.

iblaine 9y ago

VR sickness is primarily caused by latency. ie you move your head, the image takes a few milliseconds to respond and you feel dizzy. But there are other types of VR sickness, like the inability to focus on an object. This research improves your ability to focus on objects at different depths. Your vision is less blurry. So yes, this research does help eliminate nausea in VR. To say otherwise is misleading.

Sean1708 9y ago
There's a big difference between "will help reduce nausea" and "will eliminate nausea".
deleted 9y ago
[deleted]
cjslep 9y ago
> So yes, this research does help eliminate nausea in VR. To say otherwise is misleading. Are you responding to someone? Because this doesn't really make sense to me. Can you clarify?
istorical 9y ago
Primarily? I'd say it's primarily caused by a disconnect between what the human vestibular system (your sense of balance and spatial orientation) is telling the brain your body is doing vs the acceleration forces that your body is feeling vs what your eyes are telling you about what the body is doing. Or in other words - differences in movement/locomotion your eyes are seeing in HMD vs what your other senses are experiencing. Which probably would include latency as a subset.
- Balgair 9y ago
  The term in the medical literature you are looking for is 'Simulator Sickness'. There are 2 main theories on it's occurrence, one of which is the issues with the vestibular system. Wikipedia has a good introduction here: https://en.wikipedia.org/wiki/Simulator_sickness https://en.wikipedia.org/wiki/Simulator_sickness To note, simulator sickness is not that new, we have been grappling with this for ~65 years since the first flight simulators. Despite the massive funding that the DoD has at it's fingertips, we have not found a cure for it OR there has not been a lot of work done to find a cure. However, it seems that more time in the simulator does help, thought it may then hinder actual flight performance. Also, the more experienced pilots had a higher occurrence of it. Again, it's not that well understood.

DaveSapien 9y ago

This is a much better (IMO)approach by Nvidia: http://www.fudzilla.com/news/graphics/39762-nvidia-shows-off-its-light-field-vr-headset-at-vrla-2016 http://www.fudzilla.com/news/graphics/39762-nvidia-shows-off...

IshKebab 9y ago
If I'm understanding things, that only has two focal planes. They mention the approach of just having several focal planes in the video.
- randyrand 9y ago
  They time-multiplex the two planes (switching the displays on and off quickly). Amazingly, the brain interprets it as a 3rd, or 4th, or 5th, plane! This Facebook optic uses the same technique.
  - IshKebab 9y ago
    No that's not accurate. The brain may not be able to distinguish the two focal planes but it's just wrong to say "the brain interprets it as a 3rd, or 4th or 5th plane". And this Facebook technique clearly doesn't use the same technique. Instead of a focal plane they have a focal surface that isn't restricted to a plane. It might not be noticeably better (who knows) but it's definitely not the same technique.
- DaveSapien 9y ago
  Yes that seems so. Must be for cost reasons, I initially thought the new system was like the one they demoed in 2013 (http://lightfield-forum.com/light-field-camera-prototypes/nvidia-near-eye-light-field-display/ http://lightfield-forum.com/light-field-camera-prototypes/nv...), much more like the Lytro light field camera in approatch. So yes as already said, same idea as Oculus.

6stringmerc 9y ago

Ctrl+F "inner ear" - no results. To me, that's going to be the keystone to a functional VR experience. Until there's some relatively non-intrusive mechanism to fool the body's systems into playing along, I'm sorry, I don't think image resolution or refresh or FPS will solve the problem. They're all very important, sure, but I think the biology of the conundrum is the most challenging short term.

oopsies49 9y ago
How do astronauts combat nausea in space? Practice. I personally don't expect a technical solution to this any time soon. We just have to acclimatize ourselves.
- ethbro 9y ago
  > We just have to acclimatize ourselves. Aww. But I already had my scalpels out for DIY inner ear surgery. I just finished watching the YouTube howto and everything... :(
- falcolas 9y ago
  > We just have to acclimatize ourselves A lot of discomfort for relatively little (at least thus far) gain.
- shazow 9y ago
  Not everyone is fit to be an astronaut. Not everyone is able to eliminate nausea through reasonable amounts of acclimatization. The target audience for VR is everyone.
  - Houshalter 9y ago
    Everyone? It's always going to be a niche thing. There's lots of people who can't play regular video games because of motion sickness, but the video game industry doesn't suffer.
    - y4mi 9y ago

ryandamm 9y ago

I think this is overstated (though without looking through the prototype, this is only speculation). During normal, outside-of-headset vision, we focus naturally and quickly on whatever we're looking at. We don't spend time with our eyes consciously defocused on subject matter in our foveal view. So anything that's out of focus will tend to be in our peripheral view. So this is a peripheral technology. I think everyone's still looking for the killer additional tech that will make VR perfect -- but it's not about one magic tech bullet, it's about ecosystems slowly growing, and content getting better. (The headsets are better than people think.)

IshKebab 9y ago

I'm a bit skeptical of how much of a problem this really is. I have never noticed it in VR. Perhaps because: 1. Display resolution is still quite low, so really everything is blurry. 2. You will never be able to notice blurriness where you aren't focused anyway because you aren't looking there! Everything is always blurry in your peripheral vision. 3. Surely eye focus is a feedback system, like in cameras? I mean nobody has problems focusing on TVs because your eyes just magically change focal length until the image is sharp. I am stereoblind so maybe it is a big problem for others.

miketuritzin 9y ago
Have you tried to focus on something that is under 1 foot from your eyes in VR? From my experience it is virtually impossible, and I think the cause is related to this problem (not 100% sure though).
- jobigoud 9y ago
  I think for very close objects you need to have a very good match between your interpupillary distance, the interaxial distance of the HMD lenses and the interaxial distance of the virtual cameras. Also it depends on the HMD screen size, at some point parts of the object's image for each eye falls off screen.
- saltcured 9y ago
  Try closing one eye to rule out bad alignment of the viewing frustrums to your eyes' actual viewing positions and angles.
gervase 9y ago
I might be an outlier, in that I don't experience motion sickness / vestibular mismatch from VR, but I suffer from this focal issue pretty severely. When I'm in VR, I notice it slightly, but it's easy to get used to. However, as soon as I get out, I find it very difficult to focus normally again, usually for about 4x the amount of time I was in. This can make it difficult to do anything that benefits from depth perception, such as driving, walking on uneven terrain, etc. If this technique can improve that side effect, it would make quite a difference for me personally. No one else I've talked to about VR have this same experience, so it might only affect a subset of the population.

russdill 9y ago

Would it be possible to detect the focal distance of the eye and change the entire focal depth of the display to keep it always in focus, similar to automated vision testing devices? It could then perform blurring of out of focus objects as a rendering step.

deleted 9y ago
[deleted]
noio 9y ago
That wouldn't account for the fact that the lenses in your eyes are physically trying to focus on a different plane because of other cues in the scene. If you look at a close by object in VR (like say the gun you're holding) your eyes will automatically try to accommodate to render nearby objects sharp. By doing so they will actually make the gun blurrier, because it (like everything else) is rendered on a medium-distance plane, while your eyes are cued in to focus on a nearby object. Or did I get that completely wrong?
- russdill 9y ago
  That's why you'd need to have the rendering engine blur things that should not be in focus. From what I understand on how the medical systems work, you'd shine an IR light into the user's eye. The return from the light would allow you to measure the focal distance of the eye. You'd then adjust the lenses of the headset so that the display is exactly that focal distance away. The focal distance of the eyes would get passed to the renderer. The renderer would do proper focal distance bluring to objects that are not at the correct focal distance. The main limitations would likely be the measuring of eye focal distance in realtime, having lenses respond to it in real time, and the possibility that the always in focus screen door effect would cause issues.
  - russdill 9y ago
    Oh, and accommodation really helps your eyes speed up focusing. Try focusing on things near and far with both eyes, and then only one eye. You can still do it with just one eye, but it goes much faster with 2.

mhalle 9y ago

Having a display that can support ocular accommodation (selective focus by the eye) is an important research development, though it will most likely not change the viewer's experience in a radical way. Practical electronic 3D displays requires bandwidth reduction, both data bandwidth for transmission and optical bandwidth to create practical or lower-cost optical modulators. The goal is to use bandwidth reduction techniques that produce little or no visual artifacts. Some of the techniques used are the same as in 2D (spatial discretization, time multiplexing, compression), while others are unique to 3D (view discretization, limits on view angle, elimination of coherence). Head-mounted displays are basically descendants of stereoscopes, the first 3D displays developed by Wheatstone in 1838. Wheatstone's amazing discovery was that you can throw a huge amount of information about the world away, provide just two images from two viewpoints, project them out to infinity in front of a viewer's two eyes using two lenses/light paths, and a vivid sense of 3D is evoked. That's an incredible amount of information reduction from real life. In the traditional stereoscope, accommodation is thrown away, mostly because its really hard to recreate electro-mechanically, but also because we're generally fine with it. Accommodation isn't effective for distant objects (or for even larger depth ranges as we get older we lose our ability to accommodate), so we likely have neural circuitry to discount imperfect accommodation cues. One of the reasons we turn on bright lights when doing detailed work is to stop our eyes down and increase our depth of field, reducing the need for accommodation. However, there have been perennial debates about the physiological impact of conflicting depth cues involving accommodation, and those debates are more interesting in VR where objects can be (virtually) very close to the viewer and the viewer can dynamically change their physical relationship with virtual objects. Until you have a light modulator that can let you experiment with selectively modulating accommodation within a scene, you can't provide real data on how important accommodation (even approximate accommodation) is for a particular application. Can't wait to see the studies. We did some similar focal plane manipulation in holographic video more than a decade ago, for related reasons (see Fig 7): https://www.researchgate.net/publication/255603167_Reconfigurable_image_projection_holograms https://www.researchgate.net/publication/255603167_Reconfigu...

AndrewKemendo 9y ago
I would argue it's an important but not the most important thing for VR/AR given what I've seen from consumer feedback. The most important based on real world talking with consumers is a tossup between FOV and latency.

hackjam600 9y ago

+ University grades changing + Facebook hack + Email interception hack + Email accounts hack + Grade Changes hack + Website crashed hack + Word Press Blogs hack + Retrieval of lost file/documents + Erase criminal records hack + Databases hack + Sales of Dumps cards of all kinds + Untraceable Ip + Bank accounts hack + Individual computers hack + Websites hack + Control devices remotely hack + Burner Numbers hack + Verified Paypal Accounts hack + Any social media account hack + Android & iPhone Hack + server crashed hack + Text message interception hack + Twitters hack + Skype hack + Credit cards hacker + We can drop money into bank accounts. + credit score hack + blank credit card sale + Hack and use Credit Card to shop online + Monitor any phone and email address + Tap into anybody's call and monitor their conversation CONTACT:hackjam600@gmail.com

nsxwolf 9y ago

Weird that I haven't noticed this yet when using VR. I wonder if I'll notice next time after watching this.

spyder 9y ago

There is nice table in the paper which compares the capabilities of the different technologies trying to solve the DOF problem in HDMs. http://i.imgur.com/8rdoeS3.png http://i.imgur.com/8rdoeS3.png

AndrewKemendo 9y ago

Honestly I found the paper critically lacking where they attempted to make reference or comparisons to virtual retinal displays. Saying that a VRD is functionally restricted to moderate FOV in comparison to the 120 degree FOV of the Rift - using only the embodiment of the deform-able membrane mirror as reference is ridiculous on it's face. Even a rough version of the deformable mirror AR VRD described by researchers at UNC Chapel Hill [1] accomplishes 100 degrees FOV with accommodation. They went further with the Pinlight achieving 110 in 2014 [2] The technical limit according to our own work for VRD FOV is H: 200o, V: 140o (Combined). So either they're ignoring work in the field intentionally cause they don't want to do VRD or they don't know about it. My guess would be the former. [1]http://telepresence.web.unc.edu/research/dynamic-focus-augmented-reality-display/ http://telepresence.web.unc.edu/research/dynamic-focus-augme... [2]http://www.cs.unc.edu/%7Emaimone/media/pinlights_siggraph_2014.pdf http://www.cs.unc.edu/%7Emaimone/media/pinlights_siggraph_20... edit: I find this whole thing extremely frustrating. Facebook could throw 2 billion dollars at VRD tech and actually get to a working stable consumer grade system if they wanted to - everything is there for it. Why aren't they?

deleted 9y ago
[deleted]
tjoff 9y ago
Because the requirements to do it well makes it costly, which in turn makes it a gimmick.
- AndrewKemendo 9y ago
  Why would it be a gimmick if it's costly? Or are you saying the proposed thing is gimmicky?
  - tjoff 9y ago
    It won't get traction if it is costly, and without traction noone will bother developing more than demos and proof of concepts for it. And thus for the mainstream it won't have any real value. Niche markets is another thing but not lucrative enough for a big investment.

cobbzilla 9y ago

i guess whatever the faults of the paper, i like that they do have a product on the market and are demoing and publishing research. magic leap? not so much.

eutropia 9y ago

Wont this also need gaze-tracking to be successful? In their video they described a manually moved camera. Is this technology compatible with foveal rendering?

drcode 9y ago
No, I think this technique presents differently-focused areas to the eye all at once (sort of like natural light, just at extremely coarse resolution) and eye tracking is not needed.

migueloller 9y ago

"It may even let people who wear corrective lenses comfortably use VR without their glasses." If just for this, it's a move in the right direction.

whatnotests 9y ago
Yes! Many people cannot use the VR Headsets unless they wear contacts, and even then it feels quite unnatural.

highd 9y ago

Sort of strange to describe it as a "discovery" - I'm sure a team of engineers with a variety of fields of expertise spent 1-3 years solving problems that led up to this. A "discovery" would seem to describe something that existed in the aether prior to their work - it seems to diminish the innovation and effort they put in.

sitkack 9y ago

No one has mentioned magic leap.