4 ms·

I still don't quite get how Magic Leap's "Lightfield" display is different from that of other augmented reality devices. The article has a few things to say abo

I still don't quite get how Magic Leap's "Lightfield" display is different from that of other augmented reality devices. The article has a few things to say about this, but I still don't really understand: > A Dynamic Digital Lightfield is a binocular display that can project digital objects into the world such that light enters the eye as if it were reflected from a real object. Our lightfield is biomimetic (mimics our biology/physiology). Magic Leap One creates a seamless experience where digital and analog lightfields combine into a single scene. For example, with Magic Leap One, you can render digital flowers in a physical vase so they can appear fresh every day. Like that. How is that different from what Hololens does? > Abovitz told me that the light can only be interpreted by a human — not a camera — so part of his problem in launching his product is that he can't film the experience. How is a camera sufficiently different from the human eye that you physically can't film an image through the headset? Or is this just saying you can't film it in the sense that a flat image on a monitor can't really convey the experience of trying the headset on for yourself? (The latter is a well known problem with VR.) If the latter, again, how is that different from Hololens? All-in-all the product does look promising; it appears to be less bulky than competing headsets and the devkit is less expensive too. I'm just not sure whether the display technology is a competitive advantage or not.

danielvf 8y ago
The Magic Leap doesn't just dangle one image in front of your eyes, like Microsoft's Hololens. Rather, the Magic Leap projects physically projects several visual layers at different focus distances. If you are looking at something near in the real world, your eye will change focus to look at it, and the "near" projection in the Magic Leap will be in focus, while the "far" projection will go blurry, right along with the "far" in the real world. However, I think Magic Leap has reduced the number of layers down to something like three [edit: two], from the many more layers they were showing before.
- tigershark 8y ago
  in this model there are only two focal planes as far as I know.
- Ajedi32 8y ago
  It's a multifocal display? That could indeed be a significant competitive advantage. If that's the case though, I don't know why they don't just call it a multifocal display instead of using all this confusing, non-standard terminology like "Dynamic Digital Lightfield".
  - RubenSandwich 8y ago
    My best guess is marketing. Why did Apple call it's video calling FaceTime? It separates them from the competition.
  - frabcus 8y ago
    The name is meaningful and standard to me. I call it a "Light field display" - see https://en.wikipedia.org/wiki/Light_field https://en.wikipedia.org/wiki/Light_field See also the reverse "Light field camera" https://en.wikipedia.org/wiki/Light-field_camera https://en.wikipedia.org/wiki/Light-field_camera The "dynamic" part I don't understand yet - something to do with eye tracking allowing a higher resolution than otherwise would be available? Where's "multifocal display" from as terminology?
    - Ajedi32 8y ago
      Maybe by "dynamic" they mean it's actually a varifocal display, like the Oculus Half-Dome prototype? https://www.oculus.com/blog/introducing-the-team-behind-half-dome-facebook-reality-labs-varifocal-prototype/ https://www.oculus.com/blog/introducing-the-team-behind-half... As for where I heard the term "multifocal display", I'd recommend checking out Douglas Lanman's talk at SID Display Week, where he goes into detail on a whole bunch of possible solutions to vergence-accommodation conflict in VR headsets: https://www.youtube.com/watch?v=GbRvkS3GDsw https://www.youtube.com/watch?v=GbRvkS3GDsw
modzu 8y ago
i believe the light is projected onto your retina. so your eyes are still seeing light from your regular environment and then the images from the leap are superimposed. to capture what the user sees you would need to film both the light from the leap plus what the person's eye is seeing.
- Ajedi32 8y ago
  Isn't that necessarily true of all augmented reality headsets though?
golergka 8y ago
I have had an opportunity to see a demo by another stealth company (which may have been quietly acquired by Magic Leap since then). They used lasers to project light directly onto my retina. It was nothing like Magic Leap, no 3d, just test 2d images - and yet, these razor-sharp, light images that I was able to see in front of the sun, were better than any computer display that I've ever seen. I don't know how would I convince anyone about the quality of the product aside from telling them to see for themselves.
- Mahn 8y ago
  So basically, it sounds like the main advantage of the Magic Leap compared to HoloLens or existing VR headsets is likely the perceived resolution.
- ModernMech 8y ago
  You let a stealth company shoot laser beams in your eye and then you looked at the sun? Are you blind now?
  - golergka 8y ago
    I was looking into a sun-lit street and in the eyepiece and saw virtual display despite the bright background.
michaelbuckbee 8y ago
So "standard" AR glasses normally use reflective lenses (think like a two way mirror) and a separate lcd screen. This is most obvious in something like the Meta2: imagine you started with a ballcap, used double sided tape to attach a smartphone to the bottom of the rim (touching your eyebrows and screen pointed down), lastly attach a reflective piece of plastic from the forward edge of the brim angling down to around your nose. Or just look at pictures of it here (though they are all cleverly angled to not show the eyebrow screen): https://www.metavision.com/ https://www.metavision.com/ The resulting image is both digital (from the LCD) and the reality of what you're seeing around you, but by necessity it's dimmer and the digital screen images look more translucent. The promise of MagicLeap and lightfield projection is that you can get a fundamentally much more solid looking image by directly projecting light into your eye (with no intermediary lenses), though it's unclear exactly how much of this is happening in the current MagicLeap developer release. It's not a perfect analogy, but it's the difference between looking at a computer screen with and without some tinted sunglasses, or a couple of 2 way mirrors between you and your monitor.
AndrewKemendo 8y ago
The short answer is that the ML One is not a virtual retinal display/fiber scanning array as initially promised in 2012/3. They departed from that tech for this version some years ago. So they basically have the same category of tech as the hololens but with a few additions, tricks and optimizations. You're correct in that the display tech isn't the differentiator right now. They are hoping the ML One does well enough to support actually getting the FSD to market. It's iterative not revolutionary. That's not to take anything away from them, this is an exceedingly hard problem and space so it's astonishing they are getting anything out at all.
frabcus 8y ago
I don't think we know quite enough about the detail of Magic Leap's technology yet - there are some analysis articles based on their patents, but it's an incomplete picture. However, the general concept of a "light field display" is worth knowing. Normal displays work by creating pixels that glow. Each pixel emits light in all directions. That passes through your eye if it is looking at the object, which focuses it back into one point on the retina. In a (pure imaginary) light field display, instead individual rays of light are produced, not pixels. The individual rays can be faked so even though emitted very near your eye, they are focused by your eye and your brain thinks they are further away. Possible advantages of this (off the top of my head based on reasoning from the physics and the way vision works and the way existing displays are used): - Can create objects at any distance - including infinity. Your brain (it's stereoscopy tricked) will believe they really are at that distance. - The viewer can change focus, and the objects will correctly blur and come in and out of focus. - An individual large object could have varying distance and focus across its form as in the real world. - Because of the improved stereo and focus, in an AR display especially, there will be less contrast between the projected objects and the real world. They will just seem "more real" in comparison to real objects. - Vision correction. In theory a light field display could project images that correct for any vision defects you have - dynamically. (In practice the talk of special lenses for glasses whereas in the OP article shows that Magic Leap isn't at a level it can do that yet) - Special effects. So of course you can then do other funky things - play with faking visual defects for special effects. Your character could appear to need glasses, and then get them as a powerup etc. Or you could make someone's vision blur when they're tired or about to die. Or have a boss which is always out of focus in an odd way. Of course Magic Leap won't be that good. It is only digital. It will be granular both in normal resolution and in effective depth of field resolution. But still... It really does sound like it has some of the above benefits.
- themattgoldfarb 8y ago
  Even with the light field display, you would still need vision correction to see the real world objects (since it's AR).
- russdill 8y ago
  There's a really easy way to tell if something is that type of light field display and ML is clearly not. Such a display would only be visible from a single point. From the very weak way ML is using the term light field, anytime there is light, there is a light field. An LCD monitor produces a light field.
dingo_bat 8y ago
It's exactly how apple's devices are "magical" and displays are "retina".
sytelus 8y ago
One big difference (if claims are correct) is that when looking at an object your eyes can actually focus just like if they really exist. In VR, there is only one focus plane so eyes tried to focus but they brain realized there is no point. This is one of the big issue in VR to use headset more that hour because it will cause headechs.

danielvf 8y ago

The Magic Leap doesn't just dangle one image in front of your eyes, like Microsoft's Hololens. Rather, the Magic Leap projects physically projects several visual layers at different focus distances. If you are looking at something near in the real world, your eye will change focus to look at it, and the "near" projection in the Magic Leap will be in focus, while the "far" projection will go blurry, right along with the "far" in the real world. However, I think Magic Leap has reduced the number of layers down to something like three [edit: two], from the many more layers they were showing before.

tigershark 8y ago
in this model there are only two focal planes as far as I know.
Ajedi32 8y ago
It's a multifocal display? That could indeed be a significant competitive advantage. If that's the case though, I don't know why they don't just call it a multifocal display instead of using all this confusing, non-standard terminology like "Dynamic Digital Lightfield".
- RubenSandwich 8y ago
  My best guess is marketing. Why did Apple call it's video calling FaceTime? It separates them from the competition.
- frabcus 8y ago
  The name is meaningful and standard to me. I call it a "Light field display" - see https://en.wikipedia.org/wiki/Light_field https://en.wikipedia.org/wiki/Light_field See also the reverse "Light field camera" https://en.wikipedia.org/wiki/Light-field_camera https://en.wikipedia.org/wiki/Light-field_camera The "dynamic" part I don't understand yet - something to do with eye tracking allowing a higher resolution than otherwise would be available? Where's "multifocal display" from as terminology?
  - Ajedi32 8y ago
    Maybe by "dynamic" they mean it's actually a varifocal display, like the Oculus Half-Dome prototype? https://www.oculus.com/blog/introducing-the-team-behind-half-dome-facebook-reality-labs-varifocal-prototype/ https://www.oculus.com/blog/introducing-the-team-behind-half... As for where I heard the term "multifocal display", I'd recommend checking out Douglas Lanman's talk at SID Display Week, where he goes into detail on a whole bunch of possible solutions to vergence-accommodation conflict in VR headsets: https://www.youtube.com/watch?v=GbRvkS3GDsw https://www.youtube.com/watch?v=GbRvkS3GDsw

modzu 8y ago

i believe the light is projected onto your retina. so your eyes are still seeing light from your regular environment and then the images from the leap are superimposed. to capture what the user sees you would need to film both the light from the leap plus what the person's eye is seeing.

Ajedi32 8y ago
Isn't that necessarily true of all augmented reality headsets though?

golergka 8y ago

I have had an opportunity to see a demo by another stealth company (which may have been quietly acquired by Magic Leap since then). They used lasers to project light directly onto my retina. It was nothing like Magic Leap, no 3d, just test 2d images - and yet, these razor-sharp, light images that I was able to see in front of the sun, were better than any computer display that I've ever seen. I don't know how would I convince anyone about the quality of the product aside from telling them to see for themselves.

Mahn 8y ago
So basically, it sounds like the main advantage of the Magic Leap compared to HoloLens or existing VR headsets is likely the perceived resolution.
ModernMech 8y ago
You let a stealth company shoot laser beams in your eye and then you looked at the sun? Are you blind now?
- golergka 8y ago
  I was looking into a sun-lit street and in the eyepiece and saw virtual display despite the bright background.

michaelbuckbee 8y ago

So "standard" AR glasses normally use reflective lenses (think like a two way mirror) and a separate lcd screen. This is most obvious in something like the Meta2: imagine you started with a ballcap, used double sided tape to attach a smartphone to the bottom of the rim (touching your eyebrows and screen pointed down), lastly attach a reflective piece of plastic from the forward edge of the brim angling down to around your nose. Or just look at pictures of it here (though they are all cleverly angled to not show the eyebrow screen): https://www.metavision.com/ https://www.metavision.com/ The resulting image is both digital (from the LCD) and the reality of what you're seeing around you, but by necessity it's dimmer and the digital screen images look more translucent. The promise of MagicLeap and lightfield projection is that you can get a fundamentally much more solid looking image by directly projecting light into your eye (with no intermediary lenses), though it's unclear exactly how much of this is happening in the current MagicLeap developer release. It's not a perfect analogy, but it's the difference between looking at a computer screen with and without some tinted sunglasses, or a couple of 2 way mirrors between you and your monitor.

AndrewKemendo 8y ago

The short answer is that the ML One is not a virtual retinal display/fiber scanning array as initially promised in 2012/3. They departed from that tech for this version some years ago. So they basically have the same category of tech as the hololens but with a few additions, tricks and optimizations. You're correct in that the display tech isn't the differentiator right now. They are hoping the ML One does well enough to support actually getting the FSD to market. It's iterative not revolutionary. That's not to take anything away from them, this is an exceedingly hard problem and space so it's astonishing they are getting anything out at all.

frabcus 8y ago

I don't think we know quite enough about the detail of Magic Leap's technology yet - there are some analysis articles based on their patents, but it's an incomplete picture. However, the general concept of a "light field display" is worth knowing. Normal displays work by creating pixels that glow. Each pixel emits light in all directions. That passes through your eye if it is looking at the object, which focuses it back into one point on the retina. In a (pure imaginary) light field display, instead individual rays of light are produced, not pixels. The individual rays can be faked so even though emitted very near your eye, they are focused by your eye and your brain thinks they are further away. Possible advantages of this (off the top of my head based on reasoning from the physics and the way vision works and the way existing displays are used): - Can create objects at any distance - including infinity. Your brain (it's stereoscopy tricked) will believe they really are at that distance. - The viewer can change focus, and the objects will correctly blur and come in and out of focus. - An individual large object could have varying distance and focus across its form as in the real world. - Because of the improved stereo and focus, in an AR display especially, there will be less contrast between the projected objects and the real world. They will just seem "more real" in comparison to real objects. - Vision correction. In theory a light field display could project images that correct for any vision defects you have - dynamically. (In practice the talk of special lenses for glasses whereas in the OP article shows that Magic Leap isn't at a level it can do that yet) - Special effects. So of course you can then do other funky things - play with faking visual defects for special effects. Your character could appear to need glasses, and then get them as a powerup etc. Or you could make someone's vision blur when they're tired or about to die. Or have a boss which is always out of focus in an odd way. Of course Magic Leap won't be that good. It is only digital. It will be granular both in normal resolution and in effective depth of field resolution. But still... It really does sound like it has some of the above benefits.

themattgoldfarb 8y ago
Even with the light field display, you would still need vision correction to see the real world objects (since it's AR).
russdill 8y ago
There's a really easy way to tell if something is that type of light field display and ML is clearly not. Such a display would only be visible from a single point. From the very weak way ML is using the term light field, anytime there is light, there is a light field. An LCD monitor produces a light field.

dingo_bat 8y ago

It's exactly how apple's devices are "magical" and displays are "retina".

sytelus 8y ago

One big difference (if claims are correct) is that when looking at an object your eyes can actually focus just like if they really exist. In VR, there is only one focus plane so eyes tried to focus but they brain realized there is no point. This is one of the big issue in VR to use headset more that hour because it will cause headechs.