13 ms·
Mixed Reality with Passthrough
- throwuxiytayq 5y agoZuck. We don't care.
- ExitPlatosCave 5y agoThis changes everything! I would find it interesting to see how a layer on top of reality would allow non-technical people to perform technical tasks. For example I don't know a lot about car engines. I can imagine being guided through minor repairs by utilizing these types of overlays. I can imagine this kind of overlay system also to provide a big boost in diagnosis for problems and systems such as car engines or even medical diagnosis.
- go_elmo 5y agoI find it hard to imagine to use the current bw only and grainy cameras of e.g. the quest 2 for such tasks, it really doesnt feel natural. But might be easy to improve this in future generation hardware
- jonplackett 5y agoFeels like a test. I think it’s a solid bet Quest 3 will have better cameras!
- go_elmo 5y agoI'm sure they chose the ones used for better CV tracking & properties, especially in low light etc, but could be easy to add a pair of true-color ones, especially for passthrough. Just checked again and the main issue IMO currently is the low framerate / blur when using it. The movement of the hands e.g. is quite uncomfortable this way.
- blensor 5y agoIt works if the "AR" is just using the real world as a kind of backdrop and not the main visual elememt. If you are using it the other way around as a kind of real world view with only a few AR elements on top it gets old pretty quicklY
- in3d 5y agoIt’s hard to see why VR headsets with pass-through would win out over AR headsets (like the Hololens) in the long term for such uses. But I suppose if you need something right now, VR would be more available and cheaper.
- msk-lywenn 5y agoAFAIK, AR headsets are still additive displays, ie. no way of displaying black. That’s the only advantage I see in using VR headsets for AR
- dagmx 5y agoAlso waveguides have other limitations, like image fidelity, FoV, needing darker environments and significant display artifacts.
- jonplackett 5y agoOne reason is that with hololens / magic leap you can’t create black. The display is like a ‘screen’ layer in photoshop and can only add additional light, so you can’t get great contrast and anything you add to the scene has a ghost-like quality. All those demos you’ve seen of AR glasses making screens on a wall are faked. Magic leap paid a special effects house to make a lot of their ‘real’ demos. Also - field of view. Magic leap has an AWFUL field of view.
- ExitPlatosCave 5y agoI think it's interesting to think about adding elements to what we experience in reality via AR. Versus importing elements from reality into a VR experience. Based on your comments I would say one of the big advantages of importing from reality into VR would be to be able to move around without colliding into physical objects in the real world.
- jonplackett 5y agoThat would be really cool. Like a game that had virtual objects but in the same places as the physical world, but hey don’t need to be the same thing. Long term I think the idea of a computer almost permanently between us and our number one most trusted sensory organ is a fast recipe for Utopia / Dystopia.
- cheschire 5y agoYou basically described the Hololens concept from years ago. What changed? This feels like old territory and the excitement doesn’t feel justified. Edited.
- sillysaurusx 5y agoIt's a HN guideline not to accuse others of shilling without evidence. (It's such a common pattern that it makes for boring reading, among other reasons.)
- plutonorm 5y agoIt a practice platform for when the tech catches up with the dream. Anyone who isn’t excited by augmented reality tech is just.... I dunno... bored of life?
- PicassoCTs 5y agoIts also a great chance, to virtualize large swathes of the "non-touch" economy - and thus be another step stone to a carbon reduced world. No need for a plastic gnome on the front lawn, just place it. No need for birthday deco, just place it in virt, no need for other deco, just place it in virt. No need for a fancy car display, just use virt-hud, with a basic-backup.
- f6v 5y ago> Anyone who isn’t excited by augmented reality tech is just.... I dunno... bored of life? I want invisible technology, the one that does everything and I don’t have to interact with it.
- malka 5y agoI am waiting for AR lenses.
- anon_tor_12345 5y ago>Anyone who isn’t excited by augmented reality tech is just.... I dunno... bored of life? lol isn't this just the most ironic claim? i'm not excited by augmented reality exactly because i'm not bored with life (i.e. unaugmented reality).
- Iv 5y agoOh, a tech from a decade ago adapted on the closed SDK of a closed device! The fact that they have to write a paragraph promising that everything is local and that videos don't get sent online, curiously, makes me even more suspicious.
- LambdaComplex 5y agoAnd if they didn't write that paragraph? It's Facebook, so you'd be suspicious anyways (and rightfully so).
- dagmx 5y agoWhat tech from a decade ago are you talking about? This is one of the first devices,at least the first in the consumer price point, to have this functionality at this high fidelity.
- potatolicious 5y agoYeah... I don't remember any kind of low-latency high-res VR video-passthrough from 2011 (or anything even close to that year)... not to mention anything at a consumer price point. I suppose FB now falls under the usual umbrella of Apple criticism: they implemented something well, for which the possibility has been known in the industry for some time, and so we gotta slag on them for not inventing it? I continue to have strong reservations about an ad company (especially that has such a dodgy history with user privacy) running this platform, but the technical achievements from FBRL are quite (pun intended) real.
- neither_color 5y agoAs a Quest 2 owner I'd love some competition but nothing comes close to its feature set at its price point. Valve index or HP Reverb may have marginally higher resolutions, but Quest 2 is both standalone AND does PCVR/Steam completely wirelessly. For what its worth it's easier to sideload apps on a quest than an iphone, and I've gotten random android utilities to run on mine, most importantly Mullvad VPN. You can install a custom app launcher and a utility that lets you instantly access it by double clicking the side button, letting you skip all of facebook's stuff. You can browse the open app catalog here: https://sidequestvr.com/ https://sidequestvr.com/ Between the oculus store, sidequest, Steam, and dropping in android APKs the device is a lot less closed than you think. Would I prefer if it was facebook account free? Yes but I dont know who else has the cash to compete with facebook on this one.
- spdebbarma 5y agoI am so glad I invested in an Oculus Quest 2 recently. So far, I've been having so much fun just exploring the various things to do in VR. I initially bought this to explore UI/UX for VR, but with AR, it opens up the technology and my personal research to a whole slew of challenges.
- laumars 5y agoAR has been in the consumer space a lot longer than VR. In fact I’m more surprised that AR had never taken off in a big way than I was about VR. Sure we’ve seen some games like Pokemon Go take advantage of it. But AR should have been a game changer.
- Mandelmus 5y agoAR will become a game changer once the hardware is there, i.e. glasses than can be worn comfortably and anywhere. Everything before that is just practice for the software and UI.
- laumars 5y agoThe hardware has been there for a decade already. You might have AR tools already in your pocket (eg Google Translate)
- enos_feedler 5y agoI agree with the idea but not that we need comfortable glasses that we can wear anywhere. AR will be a game changer on my bicycle if my helmet can integrate it. AR will be a game changer when I'm bombing down a mountain on my snowboard, if my helmet can accommodate it. My belief is that we will get deep market adoption in these verticals before any practical general purpose AR is possible
- Closi 5y ago> But AR should have been a game changer. VR turned out to be the easier of the two to turn into a viable consumer product - mainly because you can get away with a big bulky headset for gaming around the house, but you can't wear it outside for a walk. The hardware that you could wear outside for a walk (e.g. google glass) hasn't actually been AR, and has just been like a tiny heads-up display in the corner of your eye.
- Irishsteve 5y agoDo you still need a fb login to use oculus ?
- kungito 5y agoYes, you can make a fake account just for oculus. So?
- mnd999 5y agoDo you not risk being banned for being a fake account if the Facebook AI doesn’t think you’re social mediaing enough?
- SirHound 5y agoYeah you do.
- ChuckNorris89 5y agoNo you don't (officially), I also use a throwaway account and had no issues. There were a few unrelated isolated incidents in the beginning that were heavily publicized by the media but in the end you're at the mercy of the same AI ban bots just like with YouTube, Google or any other big platform so you have the same chance of finding yourself locked out if your behavior trips up the algorithm.
- flohofwoe 5y agoI recently created a throwaway FB account for "work stuff", and the amount of spam mail I'm getting now each day from Facebook to "engage" is shocking. Also: I didn't register the account with my real name and a different email address, yet Facebook magically featured out who I am and is sending out invites to the friends on my (long dead) real Facebook account. It's all creepy AF.
- KaiserPro 5y agoI've been using a "fake" account. I don't do anything dodgy with it, and my name is mostly the same as the credit card. They are much more reluctant to kill a dormant account if you attach it to hardware.
- dukoid 5y agoCan I now get a Spaceship "lounge" overlay for my flat in "AR"?
- blensor 5y agoCheck out Custom Home Mapper on SideQuest. He was experimenting with an "AR" mode back when it still required the menu glitch to activate passthrough
- skinkestek 5y agothis page was definitely not GDPR compliant. I just gave up.
- KaiserPro 5y agoWhat did they do that wasn't compliant?
- skinkestek 5y agoOpt out instead of opt in. Opt out next to impossible.
- forgingahead 5y agoVery cool innovations coming out from the Oculus team. I just wish the headset was more comfortable! I get immense pain on my face after using it for a few minutes, it seems to be a trade-off between fitting it snugly or having it slosh around my head. The tech is great but the comfort is poor. Surprising because the Rift is decently wearable. Maybe Oculus 3 will have some strong improvements with the comfort level.
- barcoder 5y agoAll my friends with the quest upgraded the strap. Oculus doing a crappy upsale, just a cost you need to swallow
- shock-value 5y agoI got a third party halo strap for my Quest 2 and it vastly increased comfort and ability to wear it for extended periods without fatigue. Might want to look into that.
- mustacheemperor 5y agoI can second all these criticisms, as well as the advice below to upgrade the strap. I use the elite strap with the VRCover face pad and strap pad and it’s a massive improvement. I also find the VRWave strap pad makes it easy for me to run my link cable through the strap and behind my head which is also a comfort improvement in some games.
- ricardobeat 5y agoThe strap upgrade is totally worth it. Much easier to find a comfortable, good fit.
- drcode 5y agoI find wearing a VR helmet for a significant period of time to be mood-depressing. I think with AR the problem would go away, but I find it unlikely that passthrough AR will be much better than VR. I will try these new demos when I have a chance.
- ericflo 5y agoIt would have been better if we could've started experimenting with this stuff more than two years ago, but Facebook kept access locked up for their use only. I'm glad they're slowly doling out more access, but the industry is now set back by over two years, and any experimentation will now happen on Facebook's terms. Of course that's all for your protection. Of course it is.
- karmakaze 5y agoI couldn't think of a good use for a low resolution, laggy grayscale mix-in video of my real environment. The page does show one use case that I'm very interested in--showing the rough position of my hands, keyboard and mouse. If there's to be advancements in VR development environments, it has to at least have an input mechanism that's as good as what we have in the real world. Having the same in VR is as good and can be improved upon.
- devwastaken 5y agoI lie awake at night and wonder, what if valve made competitive VR hardware, instead of the steamdeck? It's clear the r&d is heavily in oculus's favor, and nobody is showing up to play. Even valves new headset is marketing fluff- "reading brain waves" doesn't work how they advertise it.
- fartcannon 5y agoValve has the advantage of not being a feeding tube for a bloated, predatory social network.
- MikusR 5y agoBecause it already is one.
- chaostheory 5y agoThe other major advantage is that they’re not publicly traded which forces you to do unsavory things to chase analyst quarterly forecasts
- andyfilms1 5y agoOculus sells their hardware at a loss, and make up for it by selling your data. Valve doesn't care about being competitive. They're not market driven. They hire the best engineers possible, and let them work on what they want. Sometimes that comes together in something beautiful (Alyx, Index) sometimes it results in something out of touch (steam machines) and sometimes it results in unending development hell.
- baby 5y ago> Oculus sells their hardware at a loss, and make up for it by selling your data. Source?
- gfodor 5y agoWhat facebook does is worse than selling your data, but they rely upon people saying they sell your data, which they can deny, to hide what they actually do, which is analyze your data to sell behavior modification products.
- jbverschoor 5y agoJittery, I can already feel RSI making a come back. Still, I’d love to have a bunch of documents open, just like paper
- yellowfish 5y agoNeat, but I still wouldn't touch oculus with a 10 foot pole especially when cameras in my home are invovled
- amelius 5y agoRemember the wars between VHS and BetaMax, and VHS won because (supposedly) it allowed porn? Well, now we could see the same thing happening. Oculus doesn't allow porn (because who in their right minds would watch it with their own personal Facebook ID tied to it, with cameras running and connected to the internet?). But a competitor that can dethrone Oculus isn't yet on the horizon. Where is our "VHS"? This isn't a complaint. Just an observation :)
- felipemnoa 5y ago>>because who in their right minds would watch it with their own personal Facebook ID tied to it, with cameras running and connected to the internet? I'm willing to bet that a sizable portion of the population would watch porn on this device even after you explain this to them.
- moron4hire 5y agoI sizable portion already do
- psyc 5y agoYou must mean interactive porn on the official store? Because there are several players on the official store, plus the browser, that play porn fine.
- istorical 5y agoI have experimented with cross-fading different 3D scenes / stereo 180 videos into each other and it's truly something that cannot be explained without experiencing it. Closest thing I've experienced to like, magic, since the first time I saw an automatic door at a grocery store as a child or my first time in an airplane.
- efnx 5y agoIf you dig the idea of using VR for productivity then you should check out the SimulaVR project, which brings wayland/X support to Linux VR. https://simulavr.com/ https://simulavr.com/
- michaelnik 5y agoI hope competitors will deliver similar hardware but with more open ecosystem, check out https://lynx-r.com/ https://lynx-r.com/!
- GuB-42 5y agoI am interested by the technical aspect. Pass-through is not as obvious than sending the camera feed to the screen. If the field of view, eye position, focus and latency are not right, the world will feel wrong. If it is way off, you are as good as blind. Close range focus is already a lost cause on current gen headsets. I don't know how tolerable the other factors are and how Oculus dealt with the problem.
- moron4hire 5y ago> Pass-through is not as obvious than sending the camera feed to the screen. Pass-through is sending the camera feed to the screen. And we've had a limited pass-through for a while in that you can configure the headset to enable it when you leave the play boundaries. You can also enable it at will with a double-tap to the headset. I've walked around my entire office floor with it, greeting people. It works fine.
- dalbasal 5y agoThe social aspects are pretty interesting. I game occasionally with my brother, who lives on a different continent... Chatting about life while shooting zombies or whatnot. One of the best parts is hanging out, post game. Basic avatars and directional sound make the whole thing feel very different to video chat. A real world setting would give this a different dimension. VR is in an interesting place. This initial hype has died down a little. Users no longer haul their headsets around for show and tell. Meanwhile, a bonafide juvenile market for hardware and content exists and incrementalism gets to do its thing. Honestly, I'm kind of surprised there aren't more entrants into the hardware space.
- reilly3000 5y agoHardware projects are massively challenging even for the most well-funded operations in the world. Something about retailing, returns, support, compliance, certifications, internationalization, contract manufacturing and the rest make it difficult to enter. VR hardware is especially problematic as you have issues like vision prescriptions, nausea, HMD related injuries/RSI, safety of people and property around the players, and all kinds of other interesting avenues of liability. It’s capital intensive, and a lot of capital got tied up in Magic Leap, which appears to be a dud. I hope I’m wrong and they flourish, but my father instilled in me a strong suspicious of all Florida based businesses and they have yet change my mind.
- rewq4321 5y agoCan someone explain how they can keep the passthrough video private? Sounds like they're saying the data is processed on a separate worker that can't access the network, but that worker obviously needs to pass data back to the main thread so that UI/content can be rendered in response to the video. Perhaps they don't allow "custom" processing by the dev? E.g. can only use FB's face/pose/etc detector? That would be quite limiting though. Maybe they only allow passing back a small amount of information per frame? That'd still make it possible to leak information over several frames (if GAN latents are anything to go by, there aren't that many bits of information in a human face, for example).
- jimmySixDOF 5y agoLike much in this announcement, the details are still to be revealed; but this is from the Developer blog : Apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors. This means raw images from device sensors are processed on-device. [1] https://developer.oculus.com/blog/mixed-reality-with-passthrough/ https://developer.oculus.com/blog/mixed-reality-with-passthr...