33 ms·

Could we make the web more immersive using a simple optical illusion?

• furyofantares 4y ago
  > And even if all that becomes possible, will users be willing to close one eye to enjoy immersive experiences? That’s a question we can’t answer. No.
• braingenious 4y ago
  No.
  • deleted 4y ago
    [deleted]
  • scambier 4y ago
    But, really, the answer to that question is "no". The illusion simply doesn't work if you have two eyes and depth perception. That's why the 3DS needs to have a stereoscopic screen on top of eye tracking, or why VR headsets have 2 screens.
• wmil 4y ago
  Now we can all look forward to the day when websites refuse to load because they want camera permissions for fake depth, shiny buttons, and other random visual effects. Also they'll analyze your face and send it to the ad tracking agency.
• k__ 4y ago
  When I read the title, I was hoping for something related to static images that look different than they are. Like, bouncing objects on grey surfaces, or dots that aren't there I would love to see these illusions explored more and used to enrich UIs in some way.
• Gordonjcp 4y ago
  Interesting thing with the targets demo - less so with the zombie - but I notice that I only needed to shut one eye for the first few seconds to make the targets "pop out", and after that it worked for ages with both my eyes open.
• ETH_start 4y ago
  Need web camera integrated into the monitor and segregated from the motherboard so that these kinds of features can be provided without the risk of the webcam being hijacked and used to spy on the user.
• PaulHoule 4y ago
  I'm fairly convinced that the immediate potential of AR is tricks like this, which will whet people's appetite for more.
  • JKCalhoun 4y ago
    Maybe. But when Apple added that kind of parallax/motion to iPhone/iPad people flipped: "It makes me feel sick." Well, some people.
    • DarknessFalls 4y ago
      Did it use eye tracking or was it just using the tilt sensors?
      • JKCalhoun 4y ago
        Tilt.
  • undersuit 4y ago
    I audibly exclaimed WTF when they walked towards the screen. I was not ready for that effect.
• psuedo_uuh 4y ago
  > Imagine, for example, opening your favorite brand’s website and being presented with a miniature virtual storefront. I would puke
• zoklet-enjoyer 4y ago
  I still remember the first time I saw the Wii head tracking video. I had no idea anything like that was possible. I don't get why it didn't take off
  • goodcjw2 4y ago
    Because it's a tough sell to put a camera right in front of people's TV. IIRC, the only product managed to do this is Portal TV from Facebook... I actually tried something like this and talked to lots of potential users about this (during 2020 pandemic). But the willingness to pay for the camera is quite low.
    • Waterluvian 4y ago
      I remember when the Xbox 360 was announced and the core idea is that you plugged all your media crap into it. Combined with Kinect, it screamed, “we just want to know what you’re doing.”
      • Am4TIfIsER0ppos 4y ago
        > drink your verification can
  • russdill 4y ago
    The article describes an important issue. For the effect to look right, you have to close one eye. I suppose if you had a stereoscopic display this would not be an issue
• bm3719 4y ago
  Can't help but notice that we keep wanting to make content that should be flat more 3D (e.g., animated 3D backgrounds, scroll filler on corporate pages, the entire metaverse), and content that could benefit from being 3D more flat (like UI buttons losing their pseudo-3D bevels and not looking like interactive widgets at all).
  • ilkke 4y ago
    Re UI, my theory is that this is partly because minimalism doesn't become dated as fast. Related, I remember that what blew me away upon seeing the 3DS for the first time was not the 3d playfield (which looked exactly as I expected), but the UI in 3D. It really felt like a glimpse into an exciting future.
    • IIAOPSW 4y ago
      Here's why minimalism has such staying power. Fundamentally, pleasing design is about repetition and consistency. Brian loves patterns, that's all there is to it. Good graphic appeal isn't a seasoning like salt which you can just sprinkle on by paying for a few better "designed" assets. Its possible to make really pleasing design out of parts that are individually seen as cheap and low quality. Its also possible to make tacky tripe out of great quality parts. The important thing is consistency and arrangement. A pleasing design is one that lends itself to file compression. Minimalism is a very easy style to pull off because the fewer elements there are the easier it is to make them all consistent, uniform, simple. It takes the least time to get it right. It can be handed off to someone else without explaining any "vision" or guidelines. None of this is to say it is bad. Minimalism is good actually. Just that minimalism isn't uniquely good. Other designs can be timeless, they just rarely are.
      • wolpoli 4y ago
        I don't think we'll be seeing a move away from minimalism for the foreseeable future. The style itself will continue to evolve (hey, we just got rounded corners back!), but it'll within the confine of CSS so that no graphical/svg assets will need to be created. We'll have minimalism until a browser maker decides to make background: golfstream-leather('brown') and felt-cloth('green') work in CSS.
• deleted 4y ago
  [deleted]
• Waterluvian 4y ago
  I don’t know any of the science behind it, but my experience with these kinds of ideas (including the 2007 wiimote target demo which I replicated and showed to family), is that they feel exhausting to use beyond a 15 min novelty. My guess is that the brain detects that the motions and such are not quite right, and like a wrong eyeglass prescription, your brain and eyes work overtime trying to figure out compensation.
  • bigmattystyles 4y ago
    I had the same thing with compiz when it first came out. I loved it in all the videos; and the first day, but I quickly switched back to plain old desktops.
    • jchw 4y ago
      I actually did use Compiz+Beryll for a fairly long while. Admittedly, the bugs were annoying, but after switching off the most gimmicky bits or at least making them less pronounced (like Wobbly Windows, lol) I found it to be quite nice to use. It felt a bit like if you were using WindowBlinds but supercharged. I later switched to i3wm and threw away desktop effects though. I wouldn't mind them, but I don't miss them either.
      • atorodius 4y ago
        Oh wow, wobbly windows. I miss them. https://youtu.be/nbCg9_YgKgM https://youtu.be/nbCg9_YgKgM For a blast from the past
        • wastholm 4y ago
          I still use (KDE Plasma's) wobbly windows, but set very low. In general in GUIs I prefer a very quick animation to an instant change, perhaps because it helps make things seem more fluid and dynamic and because it guides my eyes to where a change is happening in case I'm not already looking straight at it.
        • napsterbr 4y ago
• wodenokoto 4y ago
  The fire phone did this 7 years later with 4 front facing tracking cameras and nobody cared
  • astrange 4y ago
    The New Nintendo 3DS was based around this, and although they abandoned it I think there's still a lot of room for something like it. It's more AR than AR is.
    • wodenokoto 4y ago
      Did it do head tracking? It always seemed gimmicky to me, but I’ve heard that the Mario brothers 3D worked a lot better with the depth element than in any of the remakes (I did it enjoy it just fine on my switch)
      • astrange 4y ago
        The original 3DS didn't but the "New" one did eye tracking like this, and for both eyes since it had a 3D screen.
      • ywain 4y ago
        No, it relied on the fact that the player would ~always be looking straight at the screen at a relatively fixed distance. There was a slider you could use to set the strength of the depth effect, all the way to disabling it completely. It was fun and worked pretty well, but it was very much a gimmick. EDIT: Just saw the other comment, I didn't know the New 3DS had actual eye tracking. Neat!
      • shaunsingh0207 4y ago
        I bought a new 3ds and it was absolute bliss. The mario bros 3d game was incredibly intuitive to play, that 3ds has spoiled any 3d platformer for me. I tried playing the new 3d game on the switch and even though it has shadows, I kept missing platforms etc. I think the original 3ds's lack of tracking is what turned it off for most people, though.
    • 4y ago
• jemmyw 4y ago
  It doesn't work if 2 people are looking at the screen?
  • DrewADesign 4y ago
    It works fine but the image will only appear 3D to the person wearing the sensor because the effect is entirely based on responding to the wearer's head movement.
  • chaboud 4y ago
    It doesn’t even really work for two eyes if the viewer is close to the display. Close one eye to really nail the illusion. Think of it as the perfect approach for pirates with no friends.
    • exodust 4y ago
      Even the pirates with friends likely do their online shopping solo. Like most of us do.
  • ly3xqhl8g9 4y ago
    Perhaps the pixels could be directional [1] [2]. Not sure if vaporware or an actual product, can't find more details since the Delta airport video. [1] https://www.misappliedsciences.com/home/technology.html https://www.misappliedsciences.com/home/technology.html [2] https://www.youtube.com/watch?v=82eAnMpeNZ4 https://www.youtube.com/watch?v=82eAnMpeNZ4
• startupsfail 4y ago
  Getting the camera intrinsics problem is might be solvable with the dataset of intrinsics (assuming that the browser allows to get the camera model ).
  • lern_too_spel 4y ago
    You can pose estimation of these parameters as an optimization problem. https://sites.google.com/ttic.edu/self-sup-self-calib https://sites.google.com/ttic.edu/self-sup-self-calib
• sublinear 4y ago
  > That video is from 2007, and yet the technology being shown is so impressive that it feels like it could have been made yesterday. Really? Who thinks this in 2023? I thought it was a fun little demo back then, but that was it. Both then and now I just want AR as the default for everything and for screens to die already. We've had the tech for it since then too, but not enough demand from general consumers.
  • pugworthy 4y ago
    AR as in overlay on what you're looking at, or VR as in replace everything you're looking at?
    • sublinear 4y ago
      I didn't even mention VR and I am so beyond tired of it being brought up every time AR is. I said AR because I meant AR.
      • pugworthy 4y ago
        It would take some serious overlay resolution to replace a monitor, which just doesn't exist right now (?). If the angular size of a pixel is too large, it's essentially like being nearsighted. A person with 20/20 vision can see detail down to the eye forming an angle of 5 arc minute (1 arc minute = 1/60 of a degree).
• bawolff 4y ago
  Having it be targeted as in-browser, not to mention for shopping (?!) Is weird. But it seems like an interesting technique. I wonder if it would be more applicable to an iphone app where the hardware is more consistent.
  • GavinMcG 4y ago
    Shopify serves online vendors.
    • bawolff 4y ago
      Sure. But that doesn't totally dissolve them from the proposed usecase being non sensical.
  • cush 4y ago
    The final demo was so slow and unresponsive compared to the 2007 version
• donatj 4y ago
  Oh man, I remember this YouTube video at the top from when it was new. There was a lot of cool uses for the Wii remote when it first came out.
• metalrain 4y ago
  TrackIR is existing and mature product using the original idea. Sure it requires extra hardware, but it's amazing in first person games that support it. You get to use your good monitor, don't get dizzy like in VR and it could be quite cheap. I think implementing similar thing with just camera and computer vision is a good idea, let's make this more common thing.
  • avereveard 4y ago
    Open track with neural net tracker can do what you're asking. The default configuration is a bit wonky but it works very well after some smoothing here and there.
    • tapland 4y ago
      Benefit of trackir is computation being done partially in the device/camera (as opposed to diy clips, which I use for my flight sims). There have recently been some cameras released that don’t require ir-clips and offload the computer some, but feels =\ having a webcam requiring constant camera and/or tracking. But for games it’s great and already in use.
      • dTal 4y ago
        Is it though? My understanding was that TrackIR was nothing more than an IR webcam surrounded by IR LEDs, that also served as a dongle for the freely downloadable "driver" which contained all the tracking code. With OpenTrack, a webcam with the IR filter removed, and a 3d-printed clip, I get results indistinguishable from TrackIR. A system that uses a visual light camera and no reflector clip will inevitably be slower, as it is a much harder computer vision task. But I don't think hardware acceleration comes into it anywhere.
• bbbbbgdn 4y ago
  couple live web demos with similar approaches you can check (!need a webcam for this to work) https://cables.gl/p/nrv_-a https://cables.gl/p/nrv_-a https://cables.gl/p/2Zf0Hk https://cables.gl/p/2Zf0Hk
  • bbbbbgdn 4y ago
    I believe you need immense precision of tracking for it to look decent, cause brain notice all the imperfections and gives you motion sickness
• julienreszka 4y ago
  The original demo looks much more responsive
• davidlumley 4y ago
  If you're curious what an implementation looks like: Charlie Gerard's article[1] shows how to accomplish a similar effect in-browser. It's pretty straight forward although not sure what performance looks like in large scenes. I like the idea, but don't know if people are ready to grant webcam permissions to untrusted any websites. 1: https://charliegerard.dev/blog/interactive-frame-head-tracking https://charliegerard.dev/blog/interactive-frame-head-tracki...
• mahkeiro 4y ago
  It may work for platform like ipad that have limited number of configurations. Not sure though that Apple will be open to it.
• bobnamob 4y ago
  Cynical take: Seems like an excuse for Shopify to add eye tracking to product pages for ad analytics
  • 6177c40f 4y ago
    Unfortunately it's stuff like this that's poisoned otherwise cool new tech for me. I can't help but wonder "how will this be used to advertise to me?"
    • once_inc 4y ago
      One of the cooler takes of Bitcoin or "crypto" is that you can use native tokens to pay for visiting websites. A browser plugin combined with a lightning wallet for instance could enable a website to request a payment of any amount of Satoshis to view the site, or it will redirect you to a 402 error page. This generates a way of monetizing content without the need for ads or tracking.
      • koonsolo 4y ago
        But then you need something like Nano instead of Bitcoin, which has 0 fees. Ideal for such micro transactions.
      • dmtryshmtv 4y ago
        This is implemented in the Brave browser.
      • freddie_mercury 4y ago
        I'm not sure something that actively incentivises writers blather on endlessly, bury the lede, split their content over dozens of pages, slightly raising the price knowing that people will become densensitised to reading your per page costs, lock final review scores behind more expensive paywalls than the lower priced intro pages, and who knows what other dark patterns will be invented all while forcing me to make a determination every fifteen seconds on "is this worth the price they just asked?" is supposed to be a cool thing. If you thought recipe articles were terrible today, just wait until you have to pay 0.01 for their life story and then 0.03 for the ingredients only to discover that it will cost you another 0.15 for the actual instructions.
• sjs382 4y ago
  I once saw a digital pinball machine with this sort of head tracking and the effect was WILDLY effective. Heres a similar system: https://youtu.be/Hfz82-EaOQA https://youtu.be/Hfz82-EaOQA
• galaxyLogic 4y ago
  Less is more. We don't need 3D except in a few cases. Computer output is about information, not trying to make things look like in real life 3D. When we read a book we just need some black-and-white text. There are great applications for virtual reality for instance in medicine. But in general 3D is only needed when it is needed. If it is not needed then pushing it onto users becomes a distraction.
  • once_inc 4y ago
    Medical VR [1] is a great example of that. [1] https://www.medicalvr.eu/ https://www.medicalvr.eu/
  • imiric 4y ago
    I agree. Gaming is an obvious use case, as the article mentions. Head tracking is a great and simple way to increase immersion, without requiring VR and bulky headsets. I'm surprised more games haven't experimented with this. On PC, a barrier might be knowing the camera parameters and screen dimensions, but those can be easily calibrated, as the article shows. On consoles, however, we've had cameras for two generations now, and the technology has been mostly underutilized for gimmicky features. The only game I know that did something really interesting is Alien: Isolation on PS4. With the PS Camera, it could do head tracking and noise detection, for some very cool interactions. The microphone is also severely underutilized. There are many neat applications, especially today with AI voice recognition.
• IshKebab 4y ago
  I actually implemented this at the time because it looked so cool. Unfortunately it doesn't work nearly so well in real life as it does on a video. I don't exactly recall why but I think it's just that there are so many cues that you're looking at a screen that your brain won't accept that it's really a 3D scene. It's a bit like those painted or chalked perspective illusions. They look great in a photo or video. Not so much in real life. (I don't think it's because of stereoscopy because I am stereoblind.)
• divbzero 4y ago
  It’s curious that this post came a group in Shopify called the “Spatial Commerce Team”. I’ve often wondered why online shopping hasn’t evolved beyond galleries of thumbnail images. Today’s graphics should be able to simulate the immersive experience of browsing through a physical store—walking past shelves and riffling through merchandise.
  • permo-w 4y ago
    I'd much rather scroll through thumbnails
  • exodust 4y ago
    I think you have it backwards. Shopping evolved from the practical limitations of physical shelves, to the more convenient non-physical, searchable, browsable online store categories. There is no demand for "virtual shelves" among shoppers.
• social_quotient 4y ago
  I have a somewhat related question/thought regarding daily desktop use: is it possible to have the sound from the speakers follow the browser or window that's playing it? For example, having a podcast playing on the top right corner while a different website on another browser tab also plays sound, but with the sound source corresponding to its location, making it easier to distinguish between the two. It would also be really helpful when you have a ton of browser tabs open be able to find it with a more physical metaphor.
  • meatmanek 4y ago
    glue together web audio spatializaton and the coordinates of the window and you've got it. https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Web_audio_spatialization_basics https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_A...
• boffinism 4y ago
  No because: > for the optical illusion to work, you must place the virtual camera at the position of one of your eyes and close the other one.
  • tuukkah 4y ago
    However: > Note, however, that it’s still a really fun effect if you use the midpoint between the eyes and keep both eyes open. It just doesn’t pop the same way.
• SquareWheel 4y ago
  For the intrinsic parameters of the webcam, that is a property of the hardware and not your room, yes? If so, it seems possible to crowdsourced that information to build a database. Then for most users, it's just a matter of entering your product model rather than taking 50 checkerboard photos. It's still not convenient, but it's a much smaller hurdle to overcome.
  • tomhallett 4y ago
    Agreed. My first thought was - what percentage of the target market could you serve with coding to specs for MacBooks, which have deterministic hardware. Or if the tuning parameters exist with some range, use a traversal algorithm to find them, similar to an eye exam: “A or B? [change parameter]. A or B? [change parameter]. Repeat”
• Animats 4y ago
  That idea has been around for years, for "virtual windows". This is from 2010.[1] The illusion only works for one person at a time, so, as a window, it's not too useful. For games, as someone pointed out, it works fine.[2] Although it hasn't gone mainstream. [1] https://www.youtube.com/watch?v=Vqu9NuINKbc https://www.youtube.com/watch?v=Vqu9NuINKbc [2] https://www.trackir.com/ https://www.trackir.com/
  • deleted 4y ago
    [deleted]
  • jclarkcom 4y ago
    I replicated Jonny lees demo to see how it looks in person but soon learned the problem is one of optics. Without presenting the content at the right focal depth you don’t get past accommodation and binocular convergence conflicts. Without shutter or polarized glasses you also don’t get past binocular cues - so it looks great on a video where there is one camera view but once you add a second human eye it stops looking 3D. I started a company to try to solve these problems without using head mounted displays and we developed 3D displays that work using an array of rectangular lenses and micro displays plus fresnel lens sheets. We adopted this approach to a new product category we called virtual skylights where we replicate the look and feel of an open sky with a distance sun above you. As you walk around a room the sun will follow you in the sky across units due to the optics and there is no need for active head tracking for this particular case. The effect is really amazing even though the content in this case is super simple. Hard to experience on a 2d screen but we have some videos of it here, in the reaction video you can see how people react to seeing it the first time. https://www.innerscene.com/a7/videos https://www.innerscene.com/a7/videos
    • liendolucas 4y ago
      Hats off, not only for the product but also for pursuing and solving the problems that you mentioned! Really cool! Just curious, is the technology prohibitively costly for individuals? If so, do you have any plans to make it accessible to mainstream public?
• aaron695 4y ago
  [dead]
• Fell 4y ago
  I think the technology might be more portable if you use a somehow-worn camera looking at the screen. The screen could display a calibration pattern and the camera could analyse it. Maybe a phone app would work. I think on phones it might be possible to retrieve the intrinsic values of the camera.
• knolan 4y ago
  This seems like the kind of thing the FaceID sensors in iPad and iPhones could achieve. I’ve been impressed with its use for face painting in Procreate for example.
• urbandw311er 4y ago
  This is the perfect example of a really cool solution desperately looking for a problem
  • moremetadata 4y ago
    If you watched the link, you'd see the 3d effect, so is it looking for a problem when 3d is the holy grail of av entertainment problem?
    • urbandw311er 4y ago
      Not sure what makes you think I haven’t watched the video! I watched it in full. But I don’t agree that 3D at home is a “holy grail” - for starters I’ve not seen any clear evidence of market demand.
  • chaostheory 4y ago
    A flat screen is a problem if you’re designing a 3D object. The flat screen is also a problem if you’re trying to figure out what’s wrong with a patient with flat charts. Imo the real problem is more a lack of imagination mixed with too much fear.
• sammalloy 4y ago
  Not bad, but I’m a bigger fan of the AR portals we’ve seen all over Reddit and Instagram for the past year, the most recent of which featured a virtual portal on the beach that you could walk inside and explore. I’m pretty sure everyone here has already seen it. This would allow you to devote a spare room in your house, for example, for the virtual space (or storefront). Why we don’t already have this is very frustrating.
• cousin_it 4y ago
  I'd really like to see this combined with photogrammetry, so that in video chat you could see the other person with this kind of 3D effect (and maybe also adjusting the eyes a little bit to allow for eye contact). Could be great for work meetings and online tutoring.
  • liendolucas 4y ago
    I think this has been achieved on Google's research project Starline: https://blog.google/technology/research/project-starline/ https://blog.google/technology/research/project-starline/
• noduerme 4y ago
  This is super cool. I wish WebVR was still a thing and that it had gotten more love, better browser support and more libraries adept at handling head tracking, stereoscopy, etc. During that brief period where it was standard in Chrome, I happened to be bidding a bunch of interior architectural designs for a client, so I scrapped together a demo script in ThreeJS that let me stitch together prerendered stereo panoramic renderings, and insert some interactive 3D objects / animations inside the skylight space. I patched in support for the pointer remote from the original Oculus Rift, and just stuck our architectural renderings on a private website and mailed the client a couple Oculus Rifts as a gift to check them out. It was a big "wow" factor and landed us a lot more work. None of this is to say that it would be a great idea to build websites this way. Websites are made to convey the desired information quickly, so unless the desired information is a full 3D understanding of a space I have a hard time seeing the utility of building out a 3D space as a metaphor for information. Did that make sense? Let me say it differently: We've tried a lot of visual metaphors in my life for organizing and exploring information spaces. As it turns out, the simpler ones are usually the most easily grasped and therefore the most efficient at doing their job (i.e., remaining "invisible" and serving as a transparent container for information). Even 3D interfaces that do serve the purpose of conveying spatial relationships and art often fall into the trap of trying to do too much. This is not meant as a paean to minimalism (at all - I love clever interfaces and I appreciate when information design makes you figure out a paradigm to get the most out of interacting with it - as long as "the most" is defined as more interesting views of the data I'm trying to get at). Just to say that design should showcase content, not itself, except in the case where the content is supposed to showcase the design, and then you can go nuts.
  • meheleventyone 4y ago
    WebVR is still a thing with WebXR and at least the browser devs at Meta are pushing it forward.
• Haga 4y ago
  [dead]
• Kwpolska 4y ago
  Why did Shopify of all companies do this? Who would give Random Joe’s Shopify Store the permission to use their webcam? This looks like an absolute waste of time and money and a very impractical thing to use in general.
  • lancebeet 4y ago
    There are niches in e-commerce based around specialized tools which require more permissions and user engagement than a basic store site. In some cases, such tools account for a significant portion of the total revenue. It's not surprising at all to me that Shopify would invest in this type of technology, and I would assume that other big players are doing the same.
    • Kwpolska 4y ago
      What niches and what specialized tools are those?
• jimworm 4y ago
  One potential life-changing application I haven't yet found for eye tracking is to train monocular people to align their blind eye properly. I don't know if it's possible, but if it is at least partially successful for a monocular person's simulated depth perception, it could improve their eye contact with others and benefit their social lives immensely.
• jtwaleson 4y ago
  The article states it could be really improved by using the Iris model, but I believe the MediaPipe Iris model is embedded in the FaceMesh model. Please correct me if I'm wrong.
  • calny 4y ago
    Iris is a separate model from face mesh, but it relies on face mesh for part of the pipeline > The first step in the pipeline leverages MediaPipe Face Mesh, which generates a mesh of the approximate face geometry. From this mesh, we isolate the eye region in the original image for use in the subsequent iris tracking step. … > Face Landmark Model > The face landmark model is the same as in MediaPipe Face Mesh. You can also find more details in this paper. > Iris Landmark Model > The iris model takes an image patch of the eye region and estimates both the eye landmarks (along the eyelid) and iris landmarks (along this iris contour)… https://google.github.io/mediapipe/solutions/iris https://google.github.io/mediapipe/solutions/iris
• dalbasal 4y ago
  The techniques themselves are very cool and inspiring. That said, the reason such things do/don't take off isn't about how cool they are, generally. It's about how useful they are. Even for games, immersion and realism are secondary to gameplay. For most applications, it's hard to see how a "realistic window" is useful. A flat paper, paper-like screen is better to read off, for example. Cool tech. Definitely worth exploring gaming potential. Not easy to apply to other uses. There's no benefit to making an online shop window more like a real shop window. Catalogue is what you want in any case.
  • chaostheory 4y ago
    Just apply this to CAD or 3D modeling software. I can also see doctors having a use for this tech
    • anon115 4y ago
      at first id think this was cool but moving my head constantly isnt that gonna start hurtin?
    • anon115 4y ago
      i think keeping the 3d AR aspect of it and moving the object/mesh with a very intuitive controller for 3d sculpting instead of having to turn my head and walk around behind it to modify it is the way to go.
• deleted 4y ago
  [deleted]
• in3d 4y ago
  A couple of 3D display products are coming this year: https://www.leiainc.com/newsroom/reservations-open-for-worlds-first-ai-powered-3d-tablet-award-winning-lume-pad-2-by-leia-inc https://www.leiainc.com/newsroom/reservations-open-for-world... and https://arstechnica.com/gadgets/2023/01/asus-new-16-inch-workstation-laptops-have-3d-oled-screens/ https://arstechnica.com/gadgets/2023/01/asus-new-16-inch-wor.... As my username might suggest, I’m excited! I hope they become less niche with time.
  • hoc 4y ago
    Excellent. Thanks for those links. Toshiba and HTC had this around 2011 when the mobile OSes were really bad with no updates available, phones produced by HW companies like Toshiba who had no idea on how to support sich HW software-wise. Even the Laptop 3D screen's drivers used to break with the next larger Windows update. So I'm happy to see these devices return. I hadn't thought that this would happen anytime soon. However, theses are integrated devices again. Bit afraid that the support will be as short-lived as back then. We haven't kept our FHD 3D TVs for the same absence of updated 3D TVs. But these have unified interfaces at least (Packed HMDI Stereo and SBS or TOB for normal HDMI). So maybe a standard external devices might make more sense than those integrated tablets and laptops. Time to charge my Fuji Real3D cams once again, I guess.
• defend 4y ago
  Seems interesting especially for the gaming industry, but it seems like it'd be exceptionally difficult to handle the edgecases in the real world; for instance multiple pairs of eyeballs staring at the screen. This, alongside various privacy concerns of eyeball tracking, will likely nip this technology in the bud.
• meindnoch 4y ago
  The latency makes it feel wobbly.
• gslaller 4y ago
  Why not use a tensorflowjs pose estimation model? They mention MediaPipe Iris but didn't find any tensorflowjs model?? https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html https://storage.googleapis.com/tfjs-models/demos/posenet/cam...
• Planktonne 4y ago
  > Imagine, for example, opening your favorite brand’s website and being presented with a miniature virtual storefront. You could look at their most popular products as if you were standing on a sidewalk peering into their shop. I am imagining it, and it's worse than the current norm.
  • itissid 4y ago
    Storefront branding using "smart" displays has been a variation on the same regurgitated idea since 1908[1]. [1]https://en.wikipedia.org/wiki/Selfridges#Windows https://en.wikipedia.org/wiki/Selfridges#Windows
    • mihaic 4y ago
      Physical stores already are in 3D, the internet is a different medium that actually is composed of multiple mediums, depending on what device you're accessing the content from.
  • mihaic 4y ago
    This is exactly why I think 3D TVs died out a couple years ago: once the novelty factor fades away, you realize that it's actually hindering your initial goal.
    • meragrin_ 4y ago
      Nah, it was because the costs of the content were always significantly higher than other content.
    • chaostheory 4y ago
      Besides the cost, they died because for a large portion of the population and a large portion of the supposed 3D media, the 3D effect didn’t work. It just gave people headaches and eye strain.
      • beebeepka 4y ago
• andrewstuart 4y ago
  I’m the sort of person who loves a solution without a problem and will devote months or years to it for zero result.
  • deleted 4y ago
    [deleted]
• pluc 4y ago
  Sure. Get rid of marketing and advertising and then we can start to talk about it.
• fho 4y ago
  We use this concept to great effect in a [CAVE] where we were limited to 2d projections. The thing that many are not aware of is that stereovision is just one of many ways we perceive depth: https://en.wikipedia.org/wiki/Depth_perception https://en.wikipedia.org/wiki/Depth_perception [CAVE](https://de.wikipedia.org/wiki/Cave_Automatic_Virtual_Environment https://de.wikipedia.org/wiki/Cave_Automatic_Virtual_Environ...)
• nwoli 4y ago
  It’s possible to just convert the iris model to onnx then to tfjs btw. I got 90% the way to doing it once but basically concluded it’s entirely feasible
  • jerojero 4y ago
    Heh. This is funny. I'm not saying you're wrong but don't fall prey to a common misconception: difficulty in a project is not always linear, and the first 90% might be very simple whereas the remaining 10% of the task might be the most difficult and time consuming of the project.
• bentt 4y ago
  Phones seem like a better candidate for this. The iPhone is already equipped with a fast and accurate depth scanning front camera.
• codeulike 4y ago
  Imagine, for example, opening your favorite brand’s website and being presented with a miniature virtual storefront. You could look at their most popular products as if you were standing on a sidewalk peering into their shop. I imagined this as instructed as was simultaneously bored and depressed. Why does it always have to be about shopping.
  • riidom 4y ago
    I imagine a youtube where the ads would pause as soon as you don't look at them anymore. Like that one black mirror episode. But yes, first promote some gimmicky fun stuff to get the foot in the door and when it has arrived, show the real face.
    • hammock 4y ago
      You’ve heard of YouTube Red? Now try A Youtube Orange
    • staindk 4y ago
      "Please drink a verification can" https://imgur.com/dgGvgKF https://imgur.com/dgGvgKF
      • blamazon 4y ago
        "McDonald's!" https://mobile.twitter.com/TheStreet/status/1612888601196433434 https://mobile.twitter.com/TheStreet/status/1612888601196433...
    • VLM 4y ago
      The other way around, for sure, to distract and make you look back at the thing moving in the corner of your eye.
      • hgsgm 4y ago
        They mean that the ad won't finish until you watch it.
• totetsu 4y ago
  I briefly used a trial of an eye tracking mode for macOS. It was like having telepathy
• ergonaught 4y ago
  The web is plenty immersive when there aren’t 84738473 ads and user hostile UX layouts and popups and shifting layouts and all the other things everyone ought to already know are bad.
  • chaostheory 4y ago
    The alternative is that nearly everything worthwhile is now behind a paywall, which is the reality now. HN audience can afford the paywall. However, large number of people in world cannot. Piracy is not a sustainable solution either.
• mike_hearn 4y ago
  A bit depressing in some ways. This is a pattern I've seen repeatedly, especially in the last 10 years or so - people want to do something innovative but shoot themselves in the foot up front by insisting it has to be in the browser. Then they discover that browsers are incredibly hostile environments for doing innovation projects, and instead of concluding the obvious (we should do it with a desktop app) they either ship something far worse than their original vision, or decide it can't be done at all. This one gives off especially Stockholm Syndrome-y vibes: > You need the physical dimensions of your screen, which can’t be queried even in modern web browsers ... That’s precisely what we needed, but MediaPipe Iris isn’t exposed to JavaScript ... Despite our failure to ship this to a massive audience, we still think it’s important to take a step back and marvel at what’s currently possible in modern web browsers ... It’s hard not to be excited about the future of the web right now. Seriously? Their project failed to replicate a demo from 2007 partly because browsers don't expose the APIs they needed or even basic data you could get easily from Windows 95, yet modern web browsers are a "marvel" and "it's hard not to be excited about the future of the web right now"? How is that exciting? How is any of this a marvel? It seems the opposite of exciting to me; it shows that browsers still, after 30 years of continuous development, cannot be used to push technical boundaries. The sad thing is their project didn't have to fail. They could easily have shipped Lee's demo on Shopify sites by writing a small native app for Windows and Mac, then making it download if the user clicks "View stereoscopically". The app would have access to the screen geometry and additionally detailed data from the camera driver, which could then be used to look up the necessary data from a Shopify API where they have pre-calculated the metrics for the most popular brands. On Windows the number of clicks required would be identical to in a web browser, because whilst the user would have to click the download in their tray to start the 3D view, as a web app they'd have to click a permission button to grant webcam access anyway so it amounts to a very similar UX. (On macOS Chrome it'd take one further click, on macOS Safari it will extract a .app inside a zip for you, so the number of clicks is equivalent to Windows). It's because of repeated frustrating experiences like this with teammates in my own prior research projects that I eventually took the time to write Hydraulic Conveyor, because part of the reason devs are so reluctant to go outside the browser is that the distribution DX is traditionally quite terrible. Conveyor makes massive improvements there. The goal is to have a DX similar to shipping a (static!) website and it gets very close to that indeed. Still, even now that distribution is fixed there are other reasons why research teams dig themselves into web-shaped holes like this. Not all of them can be fixed with software: 1. Experience. According to the StackOverflow survey, half of all developers have been devs for less than 10 years. We can safely assume that only a tiny minority of them have ever written a native app for Windows or macOS. Also about 50% of devs "dread" working with C++, rising to 60% for C and a whopping 75% for Objective-C, so that's not really conducive to working with the Windows or Mac APIs (Swift and C# do better). 2. Disinterest in learning. Probably due to "resumé driven development"; we devs always have one eye on the next job and what skills will be attractive to future employers. This can cause appropriate tech to get stuck in a catch-22/local minima where people don't want to learn it because other employers aren't using it, so people don't use it, so people don't want to learn it, etc. The strongest engineers will figure out what tech is most appropriate for a project and learn it if necessary even if that skill may be a one-off for that employer, but especially for people earlier in their careers they may prefer to aggressively chase what's popular to unlock job-hop pay rises. 3. The (inaccurate) perception that people don't download desktop apps. The proliferation of popular Electron versions of websites and general apathy towards PWAs is perhaps starting to correct this but it's still an issue I see sometimes. 4. Ideological loyalty. People like the idea of community/collectively developed open source tech, for all sorts of valid reasons, but this can easily turn into blindness. A complete failure to implement a 15 year old tech demo despite using vastly more advanced technology is objectively not "exciting" or "a marvel", it's actually quite embarrassing for our whole industry. But if you're emotionally committed to seeing technological progress through the lens of an ever larger HTML5 spec and nothing else, then it may seem somehow like this is still progress.
• TootsMagoon 4y ago
  Tilt Five has a take on this. https://www.tiltfive.com/ https://www.tiltfive.com/
• ZoomZoomZoom 4y ago
  It's ironic that 15 years later, the latency between the movement of the camera and the targets has only got bigger (original video vs the first one in the "Final prototype" section).
  • pxmpxm 4y ago
    First thing I've noticed as well - feels like the same 1990s VR experience.
  • throwaway4aday 4y ago
    To be fair, they're running it entirely in a web browser using only software with no specialized hardware.
    • BHSPitMonkey 4y ago
      And probably hasn't been optimized nearly as much as it could. It's clearly meant as a fun tech demo / early exploration.
• sundvor 4y ago
  The end product looked just like what putting on my TrackIR headset and booting up DCS World is like. :) Nice write up nonetheless.
• kilgnad 4y ago
  Immersion isn't really a problem. When I buy stuff online or get money from a bank I'm not looking for immersion. What I'm looking for is a fast load and snappy ui. When I watch a movie or play a game, that's when immersion matters.
• nikanj 4y ago
  Instead, we decided to eliminate all visual cues for actionability and context. The modern UX designers hiss and scream if you suggest something as banal as 3D
• 31337Logic 4y ago
  There was a time when the "web" was text only, lightning fast, and not unrelentlessly trying to hijack my attention. For a truly unique experience (for some), let's try bringing that back and see how people feel about all that free time they get back.
• stevofolife 4y ago
  Aside from the skepticality of the proposed product & business idea, on the technical side there is tremendous value from accessible, robust and portable eye tracking. During my undergrad I was fortunate enough to work at a neuroscience lab where we built a 3D Unity game interfaced with a joystick, ECoG, and eye tracker to study attention and learning. If nothing has changed, Tobii is still the global leader in eye tracking hardware [1]. They have a wide ranging of products to choose from depending on your budget. [1] https://www.tobii.com/ https://www.tobii.com/
• nathan_compton 4y ago
  God, someone _please_ make the web more immersive!
• yafbum 4y ago
  Are there any phone apps that do this kind of head tracking using the front-facing camera? I imagine this would simplify the camera calibration setup since most phones have a known geometry relating the camera to the screen location, unlike most webcams
  • drewbeck 4y ago
    This plus tracking phone position plus ML vision to fake up 3D from the front facing camera could make for a very cool FaceTime “portal” experience
  • valine 4y ago
    Amazon built an entire phone around the idea back in 2014. They had cameras on each corner of the phone to improve the effect. https://www.theverge.com/2014/6/18/5821294/amazons-fire-phone-uses-four-cameras-to-offer-3d-perspective https://www.theverge.com/2014/6/18/5821294/amazons-fire-phon...
• amsterdorn 4y ago
  How hard would it be to implement for a more popular webcam, like the one on a Macbook?
• tsunamifury 4y ago
  There is no value in reading text and looking at flat images in 3D and there are significant reductions in usability. The reality is the fixed flat scrollable page was a huge step up from the real world reading experience. Reintroducing 3D in a 2D space in a real 3D world is beyond useless without a dramatic shift in what media is consumed. And there is no atomic element of media right now that might benefit other than Instagram adding lights-box size 3D image to its feed for ads. Wait… oh no…
• FearNotDaniel 4y ago
  This must be one of those new definitions of the word "simple" that I haven't previously encountered.
• VLM 4y ago
  Has anyone considered the supply vs demand problem or perhaps there's a better way to phrase it? Yes, that is a VERY attention grabbing presentation for consumers. Why is it dead in the market? Lets consider the SUPPLIERS. I would love to buy antique used electronic gear off ebay with a full high res 3-d model of the actual device I'm buying. Lets say its a Tek 700 series oscilloscope because I love paying insane shipping fees, or maybe I'm just insane (note WRT this being an insane hobby, I need lead-silver solder, vacuum tube tester, etc). I need full photographic high res 3-d models of that particular scope including all its "problem areas" and partially replaced 'black-beauty' capacitors (or are those a HP-only thing?) and partially replaced leaky electrolytics and toasty looking resistors and all kinds of fascinating detail, including exterior shots, and all sub-chassis inside the machine. The problem is IRL you're pragmatically honestly lucky to get a single real photo of the device you're buying; not uncommon for someone to copy a pix of someone elses device when selling you theirs. "well its close enough' and in a certain defocused abstract sense that's even true... So ... we've solved the viewer-consumer side but who's going to generate the models for most realistic apps? Like consider clothing sales online. I need a new shirt for the gym, and its cool in an abstract sense that I could see "a model" wearing my future shirt, but I'd be pretty interested in the experience of seeing "me" in the shirt as opposed to the model. Where is the data going to come from, if/when we have a way to display it? I think this is the business problem where the display tech is stuck...
• kqr 4y ago
  If this requires looking at it with one eye only anyway, would an eyepatch with something like a QR code on it help calibrate the system live?
• flohofwoe 4y ago
  TL;DR: the parallax effect only looks 'impressive' on video, because our brain inteprets it as part of the filmed environment, but when you sit directly in front of the screen the illusion falls apart (unless you use some stereoscopic optics to trick your brain again into intepreting the parallax effect as 3D). It was the perfect nerd-bait for YouTube, but not much more ;)
• Reimersholme 4y ago
  [dead]
• xwdv 4y ago
  Ah yes, an excuse to get consumers to adopt eye tracking at scale.
• deleted 4y ago
  [deleted]
• blindriver 4y ago
  Is the web being more immersive really something we want or need as a society? I'm hoping that my kids learn to reject it, even though it's been my career for 30 years.
• Groxx 4y ago
  Broadly speaking: yes! I wish we used this more, it's not hard and it looks very convincing and subtle stuff feels quite good... ... when latencies are VR-like. When it lags even a little, it feels awful. So it tends to work pretty well for tech demos like this, and not at all when exposed to the real world. Specialized hardware and software stacks can do a pretty good job - see the Fire Phone for an example there. It did quite well on technical notes, the experience was smooth and battery-efficient(-enough) and the illusion was almost fast enough to be transparent. But it failed. Unless we get this kind of thing baked into everything for years, it will continue to fail, because there will be no content. And integrated webcams and their related software continue to be utter shit. We're not close to (or even moving towards) being able to use this, unless Apple surprises everyone in their next lineup and kicks off a renaissance. ..... and that's before getting into privacy issues with exposing an API that reveals info about your face. It's a harder practical realm than it seems it should be, when people have been making very effective tech demos for decades on commodity (but specific) hardware.
• funstuff007 4y ago
  Coming from Shopify, this feels like a ZIRP (zero interest rate project). i.e. projects that need not bear any fruit in the next many years. Snark aside, this looks very cool, and I definitely see use cases in gaming (and maybe in web home pages--assuming you could get the client to turn on the web cam for an unknown site). In the article I see no mention of accelerometers. Can one get nearly the same effect on mobile merely using these and not using the camera?
  • ncr100 4y ago
    Sure - there are 3d-effect backgrounds available currently. Apple offers this as an OS feature since iOS 7: Parallax / Perspective Zoom for wallpaper. https://www.wired.com/2013/09/parallax-and-the-ios-7-wallpaper/ https://www.wired.com/2013/09/parallax-and-the-ios-7-wallpap... Adding in Selfie-cam eye tracking should be relatively doable too. Could then be added to an app, a game, or to the UI layer of a phone.
• mock-possum 4y ago
  > It’s unbelievable how much it feels like the OP-1 field is floating out of the screen. It’s almost as if you could reach out and grab it. I can certainly agree with that, it's a real solid brusspup[0]-level illusion... but I'm not sure what to do with that, practically speaking, beyond "oh that looks cool." It kind makes me wonder whether my standards for 'immersive' are significantly different than theirs. [0] https://www.youtube.com/channel/UCeQEKFH31vvD-InkTGSvCrA https://www.youtube.com/channel/UCeQEKFH31vvD-InkTGSvCrA
• sinoue 4y ago
  Let's hope Apple's webkit for their AR glasses incorporates the ability to easily look at 3D like objects from 2D pages with a simple tilt of your head.
• ath92 4y ago
  Here's a demo from about 3 years ago that does something similar to the experiment in the article: https://compassionate-murdock-82fdbb.netlify.app/ https://compassionate-murdock-82fdbb.netlify.app/ It's interesting to see that these face detection models have been getting a lot better over the past couple of years, making it easier to stable and cheap webcam-based tracking. That said, if you look closely at the videos in the original article, there's some noticeable delay between the camera moving and the program responding. The original wii controller based prototype that this was based off doesn't suffer from this delay because it used infrared LEDs. It's annoying to have to place those somewhere near your face, but once you have them there the face tracking because basically instant.
• boudin 4y ago
  There was at least TrackIR providing that back then https://en.m.wikipedia.org/wiki/TrackIR https://en.m.wikipedia.org/wiki/TrackIR I remember having tried on plane simulators back then and, even if it was cool as a technology, it wasn't really useful on a computer as you simply don't move your eyes and head enough. If you do, the screen is out of your field of vision. For this to work well, the screen would either need to be massive and curved or move as you move your head.
• thih9 4y ago
  > This is an important limitation of this technique that a lot of people don’t know about: for the optical illusion to work, you must place the virtual camera at the position of one of your eyes and close the other one.
• jsnelgro 4y ago
  I think this technique would be revolutionary for CAD or 3D modeling software. It always feels clunky trying to click, drag, orbit, fly, etc your way around a viewport, when all you really want to do is quickly peek around a corner to get some perspective