8 ms·

We Won’t See Hard AR Anytime Soon (2012)

48 points by activatedgeek 10y ago

socmag 10y ago
I think it's safe to say we won't be seeing truly immersive VR any time soon either. At least any that doesn't make you want to blow chunks, or has a compelling and broad raisondetre. It's interesting to me since I've followed VR/AR for 40 years myself, and what I can say is if you look at the money and hype that have surrounded HMD's (especially in the last couple of years), the technology is still just not there, and neither are the applications. That said, a lot of very bright people are doing very cool things with technology in the name of VR/AR. That can only be a good thing. So keep at it... Nice article, thanks for resurfacing it.
- zamalek 10y ago
  > won't be seeing truly immersive VR any time soon either. Have you demoed VR on a PC, or at least PSVR? I've heard this far too often from people who have only seen a phone HMD, which are complete and utter crap (apart from the mobility aspects - which has recently been solved for Vive).
  - socmag 10y ago
    Rift and Hololens. I get that there is an immediate "Wow" factor, but then I wander off and look at the real world, and think "Holy crap!". They are good, I get that... Not great but something. I don't understand the purpose though. The best virtual reality experiences I've ever had have been between the pages of a book, or induced by psychotropic substances. If they can fix the latency, resolution and FOV issues, reduce the size of the headsets and make them cost $150, I can see that a certain sector of people might sit at home on the couch and do whatever they do. But moving around with those things on seems like an accident waiting to happen, even more so if they get the display technology right. For AR, I can see it as a technology used in business a lot more than for the consumer. That said, as one of the comments in the article mentioned... would be great to ditch my monitors and have an ultra wide panoramic coding experience. But again, that is more of a business application. I really struggle with the fatigue aspects. And yes you are right, the phone stuff is abysmal.
    - zamalek 10y ago
      > For AR, I can see it as a technology used in business a lot more than for the consumer. I'll be the first to criticize VR for being expensive, but the Hololens is absurd - you'd think that Microsoft would have paid attention to the Glass stillbirth. Oculus pulled off $600 devkits, which gave VR a fighting chance. AR has no fighting chance right now, not at that price, not with so few devs making apps (and many of the issues in the article still apply).
      - iLoch 10y ago
        It's hardly fair to compare the technology in the Oculus to what's in HoloLens. The HoloLens is an x86 PC that fits on your head and has 5 cameras, 4 microphones and a depth sensor and lasts several hours on a single charge (which is important because it doesn't need to be plugged into a $2000 PC.) I also haven't seen anyone get sick from HoloLens or trip over anything while using one. I wouldn't be surprised if the price point of HoloLens is absurdly high for a very specific reason: to stop people from thinking it's consumer technology - it isn't - not yet. And if that's Microsoft's strategy then I'd say it's a smart move. As a HoloLens developer you have to focus your efforts on enterprise and practical applications. Not gimmicks, because there's no money in that.
      - fsloth 10y ago
        Hololens is not for consumers. It's for businesses which benefit from any off-the-shelf AR, e.g. construction, and Hololens is very much appreciated in that field. From my POV in providing software for construction industry Hololens has huge potential. Will it become a just a fad, or an irreplaceable tool, remains to be seen.
      - cr0sh 10y ago
        Personally, I don't look at current consumer VR as expensive. But this is because I have the perspective of having played around with DIY VR back in the mid-1990s. Back then, the only way you could have gotten anything -close- to what we have now, would have been to drop about $50k on hardware. Even then, it still would have been lacking. Today, you can get a much better experience in your home for under $5k. To me, that's astounding. Sadly, we still don't know what to do with it - beyond games and a few other gimmicky things. There has yet to be that "must have" application for VR - we don't have a "photoshop" or "desktop publishing" or whatever you want to call it application yet - at least for the consumer. For the consumer, it's mainly about video games, and maybe some kind of "social VR" - but that's about it, for now. Those applications (except for the "hardcore") don't justify the price. I'm not sure there is an app that could.
- illumin8 10y ago
  "If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong." - Arthur C. Clarke Please, keep this in mind... ;-)
  - samdoidge 10y ago
    What relevance does this quote have? OP did not say anything is impossible, just that it is not there yet.
  - RandomOpinion 10y ago
    "A witty saying proves nothing." --Voltaire
- Findeton 10y ago
  We'll see about that. I'm working on something you might find interesting :)
  - socmag 10y ago
    Eager to here more!
- staticelf 10y ago
  > truly immersive VR A statement like that is so subjective. What is "truly immersive"? I believe it is soon here, at least from my definition of truly immersive.
  - danielbln 10y ago
    It's similar to the moving AI goalpost. At the end of the road, it's Matrix level wetware, though I'm sure at that point there will still be people waiting for "really, actually truly immersive".
    - jessaustin 10y ago
      Actually the two senses of "goalpost moving" are opposite. The first AI researchers imagined we'd be having pleasant philosophical debates with our robot butlers by now. Instead we have a network of big iron that can simulate ten chess moves more quickly than well-practiced humans can.
  - agildehaus 10y ago
    I think my Vive is immersive, with the definition that it tricks my brain into believing I'm somewhere else to the extent any technology short of a direct neural interface could. You're never going to "truly" be immersed. You feel your carpet under your feet. You feel the weight of the headset. You don't smell anything. Nothing you pick up weighs anything. The list goes on. Every item breaks "true" immersion to a certain extent. But your brain really loves to be tricked, and the visual does this rather easily. When I'm playing Valve's Xortex 26xxx, I'm in a futuristic room fighting orbs that are shooting at me. My hand is that spaceship. I don't even really care that my FOV is smaller than normal, or that it's a little blurry due to the low resolution. It's good enough that my brain is fooled. When I take off the headset, I generally have NO CLUE where I am in my room because for the entire time my brain didn't know or care. And does it need to be "truly immersive"? No. It just needs to offer experiences that you can't find elsewhere. Try coming even close to the awesomeness of Google Earth VR, for instance.
    - 10y ago
chridal 10y ago
Article should be labeled (2012).
- gman83 10y ago
  Which is when they were still actively developing an AR system, before deciding against it: http://www.theverge.com/2013/5/18/4343382/technical-illusions-valve-augmented-reality-glasses-jeri-ellsworth-rick-johnson http://www.theverge.com/2013/5/18/4343382/technical-illusion...
kriro 10y ago
Since SciFi in relation to AR is mentioned I though a recent episode of Black Mirror I watched was pretty interesting ("Men Against Fire", S3E5). I won't spoil it but the general idea was pretty interesting/food for though with regards to uses of AR.
- rajangdavis 10y ago
  This episode was really depressing. I am glad that this show highlights potential abuses of technology to the extent highlighted in this episode as this was not something I even considered. Reading this article, it seems like we should be on the cusp of realizing this technology. Does anyone know if this will be hitting the mainstream in the next couple of years?
- iLoch 10y ago
  Likewise I believe S02E01 - the Christmas episode - shares the idea a similarly dystopian fate for AR.
- AndrewKemendo 10y ago
  Black Mirror shows AR prominently in about 1/3 of it's episodes.
jcoffland 10y ago
It seems that black pixels could be achieved using LCD technology. Even of just to augment the display.
- lucaspiller 10y ago
  There's a section about that further down, it argues the issue then is focusing on a screen 1" from your eye and the real world at the same time isn't possible.
- yaakov34 10y ago
  Not really. At least not in the simple way of having an LCD screen somewhere near the eye blocking out light from an object in a certain direction by interposing a black pixel. It seems like it should work, but it would be exactly like putting some paint on a camera lens and expecting it to block out that exact part of the picture; it won't - the lens will simply get dimmer, or if the area of paint is large, there would be darkening - vignetting - of a part of the image, with a diffuse boundary; but nothing like a sharp mask. You can easily convince yourself that this won't work for an eye. Of course, you can use your finger to block out any object - as long as you hold your finger well away from your eye. Now take a match, and try to block out objects by holding it a centimeter in front of your eye; it doesn't work - you will see a diffuse slightly darkened area, but all objects will be visible.
  - jcoffland 10y ago
    Thank you for the explanation. I now see why this won't work. It seems then that the best option would be a device which affects your neural cortex more directly.
provemewrong 10y ago
Ignoring the refresh rate, you could draw black using e-ink screens.
zznneezznnee 10y ago
Anyone know what the status is of CastAR? Haven't seen any updates for a while now...
- abritinthebay 10y ago
  Last I saw from Jeri they were working on getting dev units out to people. That was pretty recently
rb1 10y ago
But we've seen "soft" AR and it appears to be fairly popular - Pokemon Go and Ingress. If you ask players of either of those games, if they would like a "harder" - i guess this means more immersive - experience while playing these games, they'll bite your hand off. There's also father.io, which looks pretty fun, but I haven't tested that out yet. For me, in my uneducated almost ignorant understanding of the situation, AR has a massive advantage over VR in that it tends to not make you quite as motion sick.
- mstolpm 10y ago
  But every player of Pokémon Go that really tries to play the game not just for 30 minutes disables the AR camera first to save battery and because it doesn't add value. People using Pokémon Go with the AR cam enabled are young kids (it's funny) and Asian tourists snapping photos with Pokémon in front of landmarks and tourist spots. I second your remark on motion sickness. And VR cuts the user/player from the real world, which is a social annoyance outside your own living room.
  - moron4hire 10y ago
    The camera isn't the AR part. It's the real world map with game data overlaid.
- moron4hire 10y ago
  VR also doesn't tend to make the vast majority of people "quite motion sick". Instances of simsickness are widely over blown and overeported. We've run thousands of VR demos, both in the road and at our dedicated "arcade" space in DC. Not one person has reported anything more than mild discomfort, and that had been a rare minority. Simsickness is anti-VR FUD. I don't know why, but there is a small cohort of AR advocates and VR detractors who would be gleeful to see the demise of VR.
simon-am 10y ago
The Magic Leap is rumoured to be able to create pure black: "Such may be used to cancel light from the planar waveguides with respect to light from the background or real world, in some respects similar to noise canceling headphones." http://gpuofthebrain.com/blog/2016/7/22/how-magic-leap-will-work http://gpuofthebrain.com/blog/2016/7/22/how-magic-leap-will-...
- AndrewKemendo 10y ago
  Realize, that site is pure speculation and nothing has been shown or demonstrated by ML or otherwise that can "display black." In theory it's possible to build a standing wave on the retina, but to do it with pixel precision with a vrd/fsd is beyond difficult compounded worse by form factor. I disagree with the article that rendering black is necessary for wide consumer adoption of AR.
  - NegatioN 10y ago
    What makes you disagree with it? I haven't re-read this post from Abrash since it was written (so I don't remember the arguments), but I think telling us more about why you disagree can give valuable input to those of us who are not in the field. :)
    - AndrewKemendo 10y ago
      To be clear his definition of Hard AR is different from consumer adoption. So really my statement should say "I don't think we need 'Hard AR'" for AR to win with consumers. I disagree with it based on my interactions with naiive (not an epithet) consumers who are playing in AR. They are excited enough about HMD AR when shown that shadow casting and an AR version of "presence" isn't as much of a consideration whether they would adopt or not. It comes down to "does this provide value." Shadows create realism and people are flexible with realism. That said, "casting shadows" would be a huge benefit and I think it will get there eventually but there isn't a roadmap to it yet.
- kimburgess 10y ago
  Somewhat relevant from earlier today: https://news.ycombinator.com/item?id=13135735 https://news.ycombinator.com/item?id=13135735.
alkonaut 10y ago
The requirements for "Hard AR" are the same as those for "good VR" becasue it needs to use video so we need e.g. exact eye-driven focusing and super low latency.
leecarraher 10y ago
make it small enough, process fast enough, and know every light source in the frame are engineering problems... ok so why is drawing black not? seems like a total engineering problem to me, one very easily solved by having lcd embedded in the display, or e-ink, or well the host of things we've already done to solve this problem. https://www.amazon.com/Hanslin-LCD-Transparent-Alarm-Clock/dp/B00BTTDQ6G https://www.amazon.com/Hanslin-LCD-Transparent-Alarm-Clock/d...
nartam11 10y ago
I'm still having a hard time understanding why adding a black pixel doesn't work. I understand 0x008000 + 0x000000 = 0x008000. But if I were to add a drop of black paint to a drop of green paint it would surely change the color. So not sure where that analogy breaks down.
- juliangoldsmith 10y ago
  The problem is, adding a drop of black paint isn't mathematical addition. When you add two different kinds of paint together, you mix them.

48 points by activatedgeek 10y ago

I think it's safe to say we won't be seeing truly immersive VR any time soon either. At least any that doesn't make you want to blow chunks, or has a compelling and broad raisondetre. It's interesting to me since I've followed VR/AR for 40 years myself, and what I can say is if you look at the money and hype that have surrounded HMD's (especially in the last couple of years), the technology is still just not there, and neither are the applications. That said, a lot of very bright people are doing very cool things with technology in the name of VR/AR. That can only be a good thing. So keep at it... Nice article, thanks for resurfacing it.

zamalek 10y ago
> won't be seeing truly immersive VR any time soon either. Have you demoed VR on a PC, or at least PSVR? I've heard this far too often from people who have only seen a phone HMD, which are complete and utter crap (apart from the mobility aspects - which has recently been solved for Vive).
- socmag 10y ago
  Rift and Hololens. I get that there is an immediate "Wow" factor, but then I wander off and look at the real world, and think "Holy crap!". They are good, I get that... Not great but something. I don't understand the purpose though. The best virtual reality experiences I've ever had have been between the pages of a book, or induced by psychotropic substances. If they can fix the latency, resolution and FOV issues, reduce the size of the headsets and make them cost $150, I can see that a certain sector of people might sit at home on the couch and do whatever they do. But moving around with those things on seems like an accident waiting to happen, even more so if they get the display technology right. For AR, I can see it as a technology used in business a lot more than for the consumer. That said, as one of the comments in the article mentioned... would be great to ditch my monitors and have an ultra wide panoramic coding experience. But again, that is more of a business application. I really struggle with the fatigue aspects. And yes you are right, the phone stuff is abysmal.
  - zamalek 10y ago
    > For AR, I can see it as a technology used in business a lot more than for the consumer. I'll be the first to criticize VR for being expensive, but the Hololens is absurd - you'd think that Microsoft would have paid attention to the Glass stillbirth. Oculus pulled off $600 devkits, which gave VR a fighting chance. AR has no fighting chance right now, not at that price, not with so few devs making apps (and many of the issues in the article still apply).
    - iLoch 10y ago
      It's hardly fair to compare the technology in the Oculus to what's in HoloLens. The HoloLens is an x86 PC that fits on your head and has 5 cameras, 4 microphones and a depth sensor and lasts several hours on a single charge (which is important because it doesn't need to be plugged into a $2000 PC.) I also haven't seen anyone get sick from HoloLens or trip over anything while using one. I wouldn't be surprised if the price point of HoloLens is absurdly high for a very specific reason: to stop people from thinking it's consumer technology - it isn't - not yet. And if that's Microsoft's strategy then I'd say it's a smart move. As a HoloLens developer you have to focus your efforts on enterprise and practical applications. Not gimmicks, because there's no money in that.
    - fsloth 10y ago
      Hololens is not for consumers. It's for businesses which benefit from any off-the-shelf AR, e.g. construction, and Hololens is very much appreciated in that field. From my POV in providing software for construction industry Hololens has huge potential. Will it become a just a fad, or an irreplaceable tool, remains to be seen.
    - cr0sh 10y ago
      Personally, I don't look at current consumer VR as expensive. But this is because I have the perspective of having played around with DIY VR back in the mid-1990s. Back then, the only way you could have gotten anything -close- to what we have now, would have been to drop about $50k on hardware. Even then, it still would have been lacking. Today, you can get a much better experience in your home for under $5k. To me, that's astounding. Sadly, we still don't know what to do with it - beyond games and a few other gimmicky things. There has yet to be that "must have" application for VR - we don't have a "photoshop" or "desktop publishing" or whatever you want to call it application yet - at least for the consumer. For the consumer, it's mainly about video games, and maybe some kind of "social VR" - but that's about it, for now. Those applications (except for the "hardcore") don't justify the price. I'm not sure there is an app that could.
illumin8 10y ago
"If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong." - Arthur C. Clarke Please, keep this in mind... ;-)
- samdoidge 10y ago
  What relevance does this quote have? OP did not say anything is impossible, just that it is not there yet.
- RandomOpinion 10y ago
  "A witty saying proves nothing." --Voltaire
Findeton 10y ago
We'll see about that. I'm working on something you might find interesting :)
- socmag 10y ago
  Eager to here more!
staticelf 10y ago
> truly immersive VR A statement like that is so subjective. What is "truly immersive"? I believe it is soon here, at least from my definition of truly immersive.
- danielbln 10y ago
  It's similar to the moving AI goalpost. At the end of the road, it's Matrix level wetware, though I'm sure at that point there will still be people waiting for "really, actually truly immersive".
  - jessaustin 10y ago
    Actually the two senses of "goalpost moving" are opposite. The first AI researchers imagined we'd be having pleasant philosophical debates with our robot butlers by now. Instead we have a network of big iron that can simulate ten chess moves more quickly than well-practiced humans can.
- agildehaus 10y ago
  I think my Vive is immersive, with the definition that it tricks my brain into believing I'm somewhere else to the extent any technology short of a direct neural interface could. You're never going to "truly" be immersed. You feel your carpet under your feet. You feel the weight of the headset. You don't smell anything. Nothing you pick up weighs anything. The list goes on. Every item breaks "true" immersion to a certain extent. But your brain really loves to be tricked, and the visual does this rather easily. When I'm playing Valve's Xortex 26xxx, I'm in a futuristic room fighting orbs that are shooting at me. My hand is that spaceship. I don't even really care that my FOV is smaller than normal, or that it's a little blurry due to the low resolution. It's good enough that my brain is fooled. When I take off the headset, I generally have NO CLUE where I am in my room because for the entire time my brain didn't know or care. And does it need to be "truly immersive"? No. It just needs to offer experiences that you can't find elsewhere. Try coming even close to the awesomeness of Google Earth VR, for instance.
  - 10y ago

chridal 10y ago

Article should be labeled (2012).

gman83 10y ago
Which is when they were still actively developing an AR system, before deciding against it: http://www.theverge.com/2013/5/18/4343382/technical-illusions-valve-augmented-reality-glasses-jeri-ellsworth-rick-johnson http://www.theverge.com/2013/5/18/4343382/technical-illusion...

kriro 10y ago

Since SciFi in relation to AR is mentioned I though a recent episode of Black Mirror I watched was pretty interesting ("Men Against Fire", S3E5). I won't spoil it but the general idea was pretty interesting/food for though with regards to uses of AR.

rajangdavis 10y ago
This episode was really depressing. I am glad that this show highlights potential abuses of technology to the extent highlighted in this episode as this was not something I even considered. Reading this article, it seems like we should be on the cusp of realizing this technology. Does anyone know if this will be hitting the mainstream in the next couple of years?
iLoch 10y ago
Likewise I believe S02E01 - the Christmas episode - shares the idea a similarly dystopian fate for AR.
AndrewKemendo 10y ago
Black Mirror shows AR prominently in about 1/3 of it's episodes.

jcoffland 10y ago

It seems that black pixels could be achieved using LCD technology. Even of just to augment the display.

lucaspiller 10y ago
There's a section about that further down, it argues the issue then is focusing on a screen 1" from your eye and the real world at the same time isn't possible.
yaakov34 10y ago
Not really. At least not in the simple way of having an LCD screen somewhere near the eye blocking out light from an object in a certain direction by interposing a black pixel. It seems like it should work, but it would be exactly like putting some paint on a camera lens and expecting it to block out that exact part of the picture; it won't - the lens will simply get dimmer, or if the area of paint is large, there would be darkening - vignetting - of a part of the image, with a diffuse boundary; but nothing like a sharp mask. You can easily convince yourself that this won't work for an eye. Of course, you can use your finger to block out any object - as long as you hold your finger well away from your eye. Now take a match, and try to block out objects by holding it a centimeter in front of your eye; it doesn't work - you will see a diffuse slightly darkened area, but all objects will be visible.
- jcoffland 10y ago
  Thank you for the explanation. I now see why this won't work. It seems then that the best option would be a device which affects your neural cortex more directly.

provemewrong 10y ago

Ignoring the refresh rate, you could draw black using e-ink screens.

zznneezznnee 10y ago

Anyone know what the status is of CastAR? Haven't seen any updates for a while now...

abritinthebay 10y ago
Last I saw from Jeri they were working on getting dev units out to people. That was pretty recently

rb1 10y ago

But we've seen "soft" AR and it appears to be fairly popular - Pokemon Go and Ingress. If you ask players of either of those games, if they would like a "harder" - i guess this means more immersive - experience while playing these games, they'll bite your hand off. There's also father.io, which looks pretty fun, but I haven't tested that out yet. For me, in my uneducated almost ignorant understanding of the situation, AR has a massive advantage over VR in that it tends to not make you quite as motion sick.

mstolpm 10y ago
But every player of Pokémon Go that really tries to play the game not just for 30 minutes disables the AR camera first to save battery and because it doesn't add value. People using Pokémon Go with the AR cam enabled are young kids (it's funny) and Asian tourists snapping photos with Pokémon in front of landmarks and tourist spots. I second your remark on motion sickness. And VR cuts the user/player from the real world, which is a social annoyance outside your own living room.
- moron4hire 10y ago
  The camera isn't the AR part. It's the real world map with game data overlaid.
moron4hire 10y ago
VR also doesn't tend to make the vast majority of people "quite motion sick". Instances of simsickness are widely over blown and overeported. We've run thousands of VR demos, both in the road and at our dedicated "arcade" space in DC. Not one person has reported anything more than mild discomfort, and that had been a rare minority. Simsickness is anti-VR FUD. I don't know why, but there is a small cohort of AR advocates and VR detractors who would be gleeful to see the demise of VR.

simon-am 10y ago

The Magic Leap is rumoured to be able to create pure black: "Such may be used to cancel light from the planar waveguides with respect to light from the background or real world, in some respects similar to noise canceling headphones." http://gpuofthebrain.com/blog/2016/7/22/how-magic-leap-will-work http://gpuofthebrain.com/blog/2016/7/22/how-magic-leap-will-...

AndrewKemendo 10y ago
Realize, that site is pure speculation and nothing has been shown or demonstrated by ML or otherwise that can "display black." In theory it's possible to build a standing wave on the retina, but to do it with pixel precision with a vrd/fsd is beyond difficult compounded worse by form factor. I disagree with the article that rendering black is necessary for wide consumer adoption of AR.
- NegatioN 10y ago
  What makes you disagree with it? I haven't re-read this post from Abrash since it was written (so I don't remember the arguments), but I think telling us more about why you disagree can give valuable input to those of us who are not in the field. :)
  - AndrewKemendo 10y ago
    To be clear his definition of Hard AR is different from consumer adoption. So really my statement should say "I don't think we need 'Hard AR'" for AR to win with consumers. I disagree with it based on my interactions with naiive (not an epithet) consumers who are playing in AR. They are excited enough about HMD AR when shown that shadow casting and an AR version of "presence" isn't as much of a consideration whether they would adopt or not. It comes down to "does this provide value." Shadows create realism and people are flexible with realism. That said, "casting shadows" would be a huge benefit and I think it will get there eventually but there isn't a roadmap to it yet.
kimburgess 10y ago
Somewhat relevant from earlier today: https://news.ycombinator.com/item?id=13135735 https://news.ycombinator.com/item?id=13135735.

alkonaut 10y ago

The requirements for "Hard AR" are the same as those for "good VR" becasue it needs to use video so we need e.g. exact eye-driven focusing and super low latency.

leecarraher 10y ago

make it small enough, process fast enough, and know every light source in the frame are engineering problems... ok so why is drawing black not? seems like a total engineering problem to me, one very easily solved by having lcd embedded in the display, or e-ink, or well the host of things we've already done to solve this problem. https://www.amazon.com/Hanslin-LCD-Transparent-Alarm-Clock/dp/B00BTTDQ6G https://www.amazon.com/Hanslin-LCD-Transparent-Alarm-Clock/d...

nartam11 10y ago

I'm still having a hard time understanding why adding a black pixel doesn't work. I understand 0x008000 + 0x000000 = 0x008000. But if I were to add a drop of black paint to a drop of green paint it would surely change the color. So not sure where that analogy breaks down.

juliangoldsmith 10y ago
The problem is, adding a drop of black paint isn't mathematical addition. When you add two different kinds of paint together, you mix them.