10 ms·

Vision-Correcting Displays [video]

172 points by pioul 12y ago

agumonkey 12y ago
Additional details here http://newsoffice.mit.edu/2014/new-display-technology-automatically-corrects-for-vision-defects-0731 http://newsoffice.mit.edu/2014/new-display-technology-automa...
amichail 12y ago
Check this out: https://news.ycombinator.com/item?id=180224 https://news.ycombinator.com/item?id=180224
kghose 12y ago
I was told the following legend about how shoes were invented. A haughty princess wanted to leave her pristine castle and explore the world. But she found the world very dusty. She told her wise men to come up with a plan to cover the world in a lush carpet. The wise men pondered and said that this could not be done. The princess threw a fit. The wise men pondered some more. Then they said "Princess, we can not cover the world in a large carpet, but we can, however, cover your feet in a small one". And the princess was pleased. And that, kids, is how we got shoes.
- eternauta3k 12y ago
  It's not crazy to go barefoot/slippers in your home though.
- joshlegs 12y ago
  Yeah, but I don't even like shoes .... On a serious note, this does seem like a dumb idea. Why would you create a million corrective devices for the myriad objects in the real world. Why would you not rather create something small and portable that you could apply to all those objects equally? OH wait, they have that. They're called eyeglasses, I think.
zavi 12y ago
Can this be done at the software level? I.e. feature built in to OS that modifies displayed image in the same way this screen does.
- sillysaurus3 12y ago
  In the researchers’ prototype, however, display pixels do have to be masked from the parts of the pupil for which they’re not intended. That requires that a transparency patterned with an array of pinholes be laid over the screen, blocking more than half the light it emits.
  - darkmighty 12y ago
    This is required to create the intended light field, but the light field approach isn't the only one, you can do simple deconvolution also. I wonder how well that would work.
- darkmighty 12y ago
  AFAIK, it's possible yes but you would need to have the distance of the eye from the display (maybe through webcam?) and the inverse required to achieve it may yield poor results depending on your aberration. In their case they are sysnthesizing a light field which inverses the aberration everywhere, you'd be doing a inverse filter specifically for your viewpoint. I don't know how good the results would be in practice, would I would like to see someone try.
thelad 12y ago
Don't get this, what's the point in 1 device being corrected while the rest of the real world is blurry? One use case example was a guy in a car with GPS navigation. So he can then see the GPS nav but how does he drive if he can't see properly!?
- bikebot 12y ago
  You'll find out when you hit your forties and you're trying to decide whether you need glasses or just slightly longer arms.
- hosay123 12y ago
  Charity school with 25 tablets, shared among 500 poverty-stricken children in a third world country. If the correction can be adjusted in software and this can be made cheaply, it's a no-brainer application. The video indicates that it can indeed be made cheaply and adjusted in software. Sounds much simpler than trying to ensure a continuous supply of glasses for 500 energetic kids
  - Houshalter 12y ago
    Not necessarily, see this guy for example: https://www.ted.com/talks/josh_silver_demos_adjustable_liquid_filled_eyeglasses https://www.ted.com/talks/josh_silver_demos_adjustable_liqui... Although I'm skeptical that third world countries will have access to high tech vision correcting tablets but not eyeglasses.
- higherpurpose 12y ago
  There are already tablets for the kitchen, for blind people, and so on. I think the "tablets for old people" niche could be a pretty big one, and all of them could feature such displays.
- Cass 12y ago
  If someone's far-sighted, as many people above the age of forty are, they might be able to see the street perfectly well, but have trouble reading a GPS screen that's only fourty cm away from their face. And speaking as one of the people whose vision defect can't be corrected by glasses, but only by uncomfortable contact lenses, there'd be immense value to me in a computer I can use without lenses - especially since reading on a screen is basically the only thing I need the damn things for while I'm at home. For someone whose work is mostly computer-based, you might be able to get away with taking your lenses out for the entire day and only wear them for the drive to work and back.
Hominem 12y ago
I think this is a great idea, and wonder why it took so long. I'd love to be able to take my glasses off and read in bed, or take my glasses off for a few hours at work while in front of my monitor.
mattangriffel 12y ago
I've always thought it would be interesting to correct vision at the brain/neural level rather than the physical level. Can anyone comment on whether this would be possible?
- nitrogen 12y ago
  I think this happens to an extent. The poor focus of bad eyesight is effectively the same as a spatial low-pass filter, and based on my own perceptions, I think the visual processing chain applies sharpening or spatial equalization to try to compensate. However, it's not enough; it can't actually restore detail that was never provided to the rods and cones in the first place.
- nacs 12y ago
  If the input (vision) is not corrected before it enters the eye then anything done afterwards, like at the neural level, would likely be similar to image post-processing filters in image editors. By no means an expert in this area but I would think the best that could be done with this type of post processing would likely amount to applying a highly specialized "Find Edges"/"Sharpen" filter in Photoshop to a blurred photo (except at a really high resolution and at > 60 frames per second). An interesting possibility with that kind of neural-level post-processing could be an on-demand digital-zoom effect so you could do a 2X-32X zoom on a faraway road sign or to "zoom in" to something really close at a macroscopic level (for surgeons/jewelers).
- kghose 12y ago
  This is indeed an interesting question. Why can't the brain develop a reverse blur function? We can do this with algorithms (http://en.wikipedia.org/wiki/Adaptive_optics http://en.wikipedia.org/wiki/Adaptive_optics). There was a fad (I guess) once about correcting your vision with practice (http://en.wikipedia.org/wiki/Bates_method http://en.wikipedia.org/wiki/Bates_method). I guess this is a limitation of the plasticity of our brains. People who have hearing or vision or other losses as young children adapt faster and better than people who have these losses when older. It would be interesting to understand if young children with vision deficits can learn to see better with time. My personal recollection is that I had NO idea things were blurry until my first visit to the optometrist when they put glasses on me. Things were so SHARP! I think the core of this is that out adaptability depends on sensori-motor loops. We can calibrate our responses for faulty sensors, but we don't correct just for the sake of correctness.
judk 12y ago
Sounds like the old "Magic Eye" pictures
DenisM 12y ago
If I understand this correctly, this is a light-field display. The implications are bigger than vision correction - LF display can reconstruct actual 3d images, as opposed to the stereo images being marketed as "3d" today. Stereo displays give two different pictures to two different eyes, but the don't provide perspective shift (the picture doesn't change when you move your head left and right), and they don't provide different focal planes (your eyes focus on the screen plane regardless of how far the object is supposed to be, creating a dissonance between the distance inferred from the angle between the eyes and the focusing distance).
- bd 12y ago
  Yup, I wonder if this technology could be used to improve head-mounted displays, to be able to focus on different parts of the virtual scene, not just being always focused on infinity (probably together with high-precision eye-tracking to figure out depth where you try to look at).
  - mkeblx 12y ago
    It can and will in relatively short order: https://www.youtube.com/watch?v=deI1IzbveEQ https://www.youtube.com/watch?v=deI1IzbveEQ Douglas Lanman, the researcher behind this technology work at Nvidia was hired a few months ago by Oculus VR.
  - nobbis 12y ago
    Magic Leap got $50M first round funding to do just that: http://www.nytimes.com/2014/07/15/science/taking-real-life-sickness-out-of-virtual-reality.html http://www.nytimes.com/2014/07/15/science/taking-real-life-s...
nitrogen 12y ago
I wonder if there's an analogy to be drawn between the pinhole mask they use and the lithography masks used to etch ICs on silicon.
serf 12y ago
One can imagine a pair of normal eyeglasses which have an IMU or accelerometer of some sort in them. When they sense they are being taken off, the users' phone switches profiles, the screen blurs, and the user moves seamlessly from glasses to phone without ever realizing what took place. Would be a neat touch. I like connected appliances that aren't. (if that makes sense)
asadjb 12y ago
I'd love for this to become common, since I have worn glasses for near sightedness since childhood. However, the biggest problem I see with this is that it is customized on a per user basis. So, while I can use my phone just fine, you can't cause it's calibrated to correct my vision deficiency. I guess that it can be used on something extremely personal; like a phone, but I don't see it becoming main stream for most displays, like a shared tablet or a computer or something.
gone35 12y ago
Link to the paper and supplemental information: http://web.media.mit.edu/~gordonw/VisionCorrectingDisplay/ http://web.media.mit.edu/~gordonw/VisionCorrectingDisplay/
AlexMuir 12y ago
http://chronicle.nytlabs.com/?keyword=germany.vietnam.iraq.russia http://chronicle.nytlabs.com/?keyword=germany.vietnam.iraq.r...
gambler 12y ago
Cool, but I think most of the people with glasses wouldn't want to rely on gadgets even more. If you can see display, but can't see physical controls or paper, that's an issue waiting to happen.
restalis 12y ago
It's an interesting idea, but even for the narrow use-case of looking at a display it does not replace for everyone the need of wearing glasses/lenses. This kind of display may only "fix" something that affects both eyes in an equal measure. If the image distortion have to be different from one eye to another, then an individual-eye-level correction is needed.

172 points by pioul 12y ago

Additional details here http://newsoffice.mit.edu/2014/new-display-technology-automatically-corrects-for-vision-defects-0731 http://newsoffice.mit.edu/2014/new-display-technology-automa...

amichail 12y ago

Check this out: https://news.ycombinator.com/item?id=180224 https://news.ycombinator.com/item?id=180224

kghose 12y ago

I was told the following legend about how shoes were invented. A haughty princess wanted to leave her pristine castle and explore the world. But she found the world very dusty. She told her wise men to come up with a plan to cover the world in a lush carpet. The wise men pondered and said that this could not be done. The princess threw a fit. The wise men pondered some more. Then they said "Princess, we can not cover the world in a large carpet, but we can, however, cover your feet in a small one". And the princess was pleased. And that, kids, is how we got shoes.

eternauta3k 12y ago
It's not crazy to go barefoot/slippers in your home though.
joshlegs 12y ago
Yeah, but I don't even like shoes .... On a serious note, this does seem like a dumb idea. Why would you create a million corrective devices for the myriad objects in the real world. Why would you not rather create something small and portable that you could apply to all those objects equally? OH wait, they have that. They're called eyeglasses, I think.

zavi 12y ago

Can this be done at the software level? I.e. feature built in to OS that modifies displayed image in the same way this screen does.

sillysaurus3 12y ago
In the researchers’ prototype, however, display pixels do have to be masked from the parts of the pupil for which they’re not intended. That requires that a transparency patterned with an array of pinholes be laid over the screen, blocking more than half the light it emits.
- darkmighty 12y ago
  This is required to create the intended light field, but the light field approach isn't the only one, you can do simple deconvolution also. I wonder how well that would work.
darkmighty 12y ago
AFAIK, it's possible yes but you would need to have the distance of the eye from the display (maybe through webcam?) and the inverse required to achieve it may yield poor results depending on your aberration. In their case they are sysnthesizing a light field which inverses the aberration everywhere, you'd be doing a inverse filter specifically for your viewpoint. I don't know how good the results would be in practice, would I would like to see someone try.

thelad 12y ago

Don't get this, what's the point in 1 device being corrected while the rest of the real world is blurry? One use case example was a guy in a car with GPS navigation. So he can then see the GPS nav but how does he drive if he can't see properly!?

bikebot 12y ago
You'll find out when you hit your forties and you're trying to decide whether you need glasses or just slightly longer arms.
hosay123 12y ago
Charity school with 25 tablets, shared among 500 poverty-stricken children in a third world country. If the correction can be adjusted in software and this can be made cheaply, it's a no-brainer application. The video indicates that it can indeed be made cheaply and adjusted in software. Sounds much simpler than trying to ensure a continuous supply of glasses for 500 energetic kids
- Houshalter 12y ago
  Not necessarily, see this guy for example: https://www.ted.com/talks/josh_silver_demos_adjustable_liquid_filled_eyeglasses https://www.ted.com/talks/josh_silver_demos_adjustable_liqui... Although I'm skeptical that third world countries will have access to high tech vision correcting tablets but not eyeglasses.
higherpurpose 12y ago
There are already tablets for the kitchen, for blind people, and so on. I think the "tablets for old people" niche could be a pretty big one, and all of them could feature such displays.
Cass 12y ago
If someone's far-sighted, as many people above the age of forty are, they might be able to see the street perfectly well, but have trouble reading a GPS screen that's only fourty cm away from their face. And speaking as one of the people whose vision defect can't be corrected by glasses, but only by uncomfortable contact lenses, there'd be immense value to me in a computer I can use without lenses - especially since reading on a screen is basically the only thing I need the damn things for while I'm at home. For someone whose work is mostly computer-based, you might be able to get away with taking your lenses out for the entire day and only wear them for the drive to work and back.

Hominem 12y ago

I think this is a great idea, and wonder why it took so long. I'd love to be able to take my glasses off and read in bed, or take my glasses off for a few hours at work while in front of my monitor.

mattangriffel 12y ago

I've always thought it would be interesting to correct vision at the brain/neural level rather than the physical level. Can anyone comment on whether this would be possible?

nitrogen 12y ago
I think this happens to an extent. The poor focus of bad eyesight is effectively the same as a spatial low-pass filter, and based on my own perceptions, I think the visual processing chain applies sharpening or spatial equalization to try to compensate. However, it's not enough; it can't actually restore detail that was never provided to the rods and cones in the first place.
nacs 12y ago
If the input (vision) is not corrected before it enters the eye then anything done afterwards, like at the neural level, would likely be similar to image post-processing filters in image editors. By no means an expert in this area but I would think the best that could be done with this type of post processing would likely amount to applying a highly specialized "Find Edges"/"Sharpen" filter in Photoshop to a blurred photo (except at a really high resolution and at > 60 frames per second). An interesting possibility with that kind of neural-level post-processing could be an on-demand digital-zoom effect so you could do a 2X-32X zoom on a faraway road sign or to "zoom in" to something really close at a macroscopic level (for surgeons/jewelers).
kghose 12y ago
This is indeed an interesting question. Why can't the brain develop a reverse blur function? We can do this with algorithms (http://en.wikipedia.org/wiki/Adaptive_optics http://en.wikipedia.org/wiki/Adaptive_optics). There was a fad (I guess) once about correcting your vision with practice (http://en.wikipedia.org/wiki/Bates_method http://en.wikipedia.org/wiki/Bates_method). I guess this is a limitation of the plasticity of our brains. People who have hearing or vision or other losses as young children adapt faster and better than people who have these losses when older. It would be interesting to understand if young children with vision deficits can learn to see better with time. My personal recollection is that I had NO idea things were blurry until my first visit to the optometrist when they put glasses on me. Things were so SHARP! I think the core of this is that out adaptability depends on sensori-motor loops. We can calibrate our responses for faulty sensors, but we don't correct just for the sake of correctness.

judk 12y ago

Sounds like the old "Magic Eye" pictures

DenisM 12y ago

If I understand this correctly, this is a light-field display. The implications are bigger than vision correction - LF display can reconstruct actual 3d images, as opposed to the stereo images being marketed as "3d" today. Stereo displays give two different pictures to two different eyes, but the don't provide perspective shift (the picture doesn't change when you move your head left and right), and they don't provide different focal planes (your eyes focus on the screen plane regardless of how far the object is supposed to be, creating a dissonance between the distance inferred from the angle between the eyes and the focusing distance).

bd 12y ago
Yup, I wonder if this technology could be used to improve head-mounted displays, to be able to focus on different parts of the virtual scene, not just being always focused on infinity (probably together with high-precision eye-tracking to figure out depth where you try to look at).
- mkeblx 12y ago
  It can and will in relatively short order: https://www.youtube.com/watch?v=deI1IzbveEQ https://www.youtube.com/watch?v=deI1IzbveEQ Douglas Lanman, the researcher behind this technology work at Nvidia was hired a few months ago by Oculus VR.
- nobbis 12y ago
  Magic Leap got $50M first round funding to do just that: http://www.nytimes.com/2014/07/15/science/taking-real-life-sickness-out-of-virtual-reality.html http://www.nytimes.com/2014/07/15/science/taking-real-life-s...

nitrogen 12y ago

I wonder if there's an analogy to be drawn between the pinhole mask they use and the lithography masks used to etch ICs on silicon.

serf 12y ago

One can imagine a pair of normal eyeglasses which have an IMU or accelerometer of some sort in them. When they sense they are being taken off, the users' phone switches profiles, the screen blurs, and the user moves seamlessly from glasses to phone without ever realizing what took place. Would be a neat touch. I like connected appliances that aren't. (if that makes sense)

asadjb 12y ago

I'd love for this to become common, since I have worn glasses for near sightedness since childhood. However, the biggest problem I see with this is that it is customized on a per user basis. So, while I can use my phone just fine, you can't cause it's calibrated to correct my vision deficiency. I guess that it can be used on something extremely personal; like a phone, but I don't see it becoming main stream for most displays, like a shared tablet or a computer or something.

gone35 12y ago

Link to the paper and supplemental information: http://web.media.mit.edu/~gordonw/VisionCorrectingDisplay/ http://web.media.mit.edu/~gordonw/VisionCorrectingDisplay/

AlexMuir 12y ago

http://chronicle.nytlabs.com/?keyword=germany.vietnam.iraq.russia http://chronicle.nytlabs.com/?keyword=germany.vietnam.iraq.r...

gambler 12y ago

Cool, but I think most of the people with glasses wouldn't want to rely on gadgets even more. If you can see display, but can't see physical controls or paper, that's an issue waiting to happen.

restalis 12y ago

It's an interesting idea, but even for the narrow use-case of looking at a display it does not replace for everyone the need of wearing glasses/lenses. This kind of display may only "fix" something that affects both eyes in an equal measure. If the image distortion have to be different from one eye to another, then an individual-eye-level correction is needed.