17 ms·

A good friend of mine at Google is the technical lead of this project (he has a Phd Princeton, and was a professor before joining Google). I've tried it in per

• bullfightonmars 5y ago
  If this was a consumer product I would buy two of these today, one for myself, and one for my parents. It is more compelling that any product I have seen for a long time.
  • paxys 5y ago
    I imagine it isn't (yet) because of the price tag.
    • bostonvaulter2 5y ago
      And it would probably be very difficult to install at the moment as well.
  • ipsum2 5y ago
    How much would you pay? I expect the lower end to be around $20,000 from hardware alone, plus at least gigabit down/up connectivity.
    • throwaway3699 5y ago
      I doubt you need gigabit for an 8K stream. The real issue is the abysmal common network infrastructure. Latency will be a bottleneck if most ISPs don't care about treating your packets properly.
      • cogman10 5y ago
        Agreed, 10Mbps is likely where it'll be, probably less. If I were to guess, they are going to apply a lot of smarts to texture mapping to avoid needing a lot of bandwidth.
        • IshKebab 5y ago
          You can't even do basic video calling on a 10 Mbps connection. You definitely need the reliability and low latency of fibre for this.
          • deleted 5y ago
            [deleted]
        • kbenson 5y ago
          Doubtful, IMO. The fact that you need it to be both extremely low latency and with essentially zero compression artifacts (probably lossless, based on their goal). If the numbers listed here[1] are correct, then the most efficient lossy codec at that time was doing 100 minutes of 8k video in ~37GB of data. From that we can intuit that it was using an average of 50 Mbps for that 100 minutes of video. For the most efficient codec, and I'm not even sure from that whether it was using lossy or lossless numbers, because apparently HVEC can do both (but I would assume lossy, since it's about streaming video in that case). You can't do weird texture mapping or lossy compression and expect people to really seem like they are there. Even if you don't notice that stuff normally watching a video, I think you'll notice it when you're interacting like someone is really in front of you, and that will throw off the immersion. 1: https://www.quora.com/In-regards-to-filesize-how-big-is-1-minute-hour-of-8K-Video-with-the-H264-HEVC-Codecs/answer/Rudy-Triana-1 https://www.quora.com/In-regards-to-filesize-how-big-is-1-mi...
    • signal11 5y ago
      I suspect eventually this will percolate down, somewhat like Facebook's Portal TV (£149) but possibly also integrated with something like a Google Assistant/Siri type smart device to power the software and do other smart-home type things in your home. Many ordinary folk already have 4k TVs at home and 8k will probably become commonplace in the future. The real bottleneck will be good low-latency broadband both up and down, but fibre to the home should make that easier. I wonder if ISPs in the future will offer QoS guarantees to enable really good videoconferencing? I mean, Zoom, Meet et al are much better than video conferencing solutions like Webex even from a few years ago, but it's hilarious how much drama there still is around video calls. "oh, sorry, I didn't realise I had muted myself", "can you hear me...?", "I think we just lost Steve", and so on. I'll be glad to see all of that just go away.
      • eps 5y ago
        Facebook Telescreen™
        • kbenson 5y ago
          To be followed up by Facebook Telepresence™ and then Facebook Omnipresence™? ;)
          • dane-pgp 5y ago
            Mark Zuckerberg might announce that, but I think he'll go straight from Facebook Telescreen™ to Facebook Panopticon™.
    • fnord77 5y ago
      probably built into smartphones in 10 years.
      • cocoggu 5y ago
        I think one of the major aspects of this technology is that the person you are talking with appears in its real size, with the same ratio than in real life. Something you cannot do with a smartphone, except if you are referring to a projector.
    • bullfightonmars 5y ago
      Well that wouldn't be a consumer product then. But I would pay as much as a high end tv. 2500? Maybe more? The experience looks to be transformative.
    • true_religion 5y ago
      Max I'd pay would be $6,000 because I'd need to buy two: one for me, and one for my parents. If it were $3000, I'd have already bought it. $20,000 is a bit out of the reach of most people, and reserves this for business use or desperate need. I could see a bunch of execs getting this installed in their home offices as a company perk, and then using it for personal reasons too.
      • ghaff 5y ago
        A lot of execs don’t have optimal setups with current tech. A lot of people just don’t care.
      • benhurmarcel 5y ago
        > reserves this for business use Even for business use a price with 5 digits would make it confined to a few executive offices.
  • codeecan 5y ago
    https://electronics.sony.com/spatial-reality-display/p/elfsr1 https://electronics.sony.com/spatial-reality-display/p/elfsr...
    • chewxy 5y ago
      This is a 3D monitor. Not a light field display like Starline. The traditional glasses free 3d monitors rely on special coatings on the glass to "split" the direction of a pixel. Some coatings are electronically controlled (like Sony's), some are physical (lenticular filters) It still displays a 2D projection of a 3D image, but twice. By contrast, my understanding is lightfield displays projects the entire volume. So you don't have a pixel, you have a voxel. So each eye gets a different bit of the same voxel
      • deleted 5y ago
        [deleted]
      • mhalle 5y ago
        The term light field display includes lenticulars and parallax barrier displays (and holograms and volumetric displays). They are all different ways of carving up, sweeping out, or multiplexing a light field.
  • CountHackulus 5y ago
    I've used the Echo Show 10 with the drop-in feature for my mom with vision issues (basically can't use touchscreens) and it works fairly well. That said I fully agree with you that something like this would be really excellent. A full-sized person's face and head in 3D is far easier to recognize and understand than a poorly lit 2D 10" display.
• somebodythere 5y ago
  Sounds super cool. I can't wait for LFD tech to become more accessible to consumers.
  • ortusdux 5y ago
    I've been thinking about pre-ordering the Looking glass portrait. $250 for the unit, and I already have a leap motion laying around. https://lookingglassfactory.com/product/portrait https://lookingglassfactory.com/product/portrait
    • bsanr2 5y ago
      It's neat and I'm waiting to see what creative things people come up with for it. A Japanese developer put together a Wizardry/Megaten-style dungeon-crawler demo: https://twitter.com/mizzmayo/status/1394171128491487234 https://twitter.com/mizzmayo/status/1394171128491487234
• Valgrim 5y ago
  It sounds like they are using this technology or something similar: https://lookingglassfactory.com/8k https://lookingglassfactory.com/8k
  • deleted 5y ago
    [deleted]
  • hathawsh 5y ago
    Does anyone know whether they're using that tech, something similar, or something else? I've always been interested in light field technology.
  • deleted 5y ago
    [deleted]
• jayd16 5y ago
  Does it have perceptible regions where the view angles are ideal? If it's as seamless as it looks in the video that would be truly novel and exciting.
• amelius 5y ago
  Was this display developed by Google, or did they buy it from elsewhere? Is it still on the market?
  • shard 5y ago
    I recall there being glasses-less 3D displays about 10 years ago, when the TV industry was trying to make 3D displays the next new thing. I wonder if this is the same technology.
    • bsanr2 5y ago
      IIRC those were displays that used sterescopy, while these simulate a light field instead.
• patall 5y ago
  Interesting. Does the light field work only for one person or multiple (they show mother and baby in the video)?
• jlebar 5y ago
  My mind was also blown when I got to demo this a few years ago. You sit down and you forget that there's technology happening. The person is there, in front of you. I don't know how else to describe it. The testimonials in the video aren't exaggerating compared to my experience.
  • rewq4321 5y ago
    Do you know if it supports multiple people on "screen" at once? Or does it rely on eye tracking of a single person (plus projection of some sort?) to be able to achieve the 3D effect?
    • ehsankia 5y ago
      I don't know anything more than what's been released, but from my understanding, light field display don't rely on eye tracking, and you can have multiple people looking from different angles (although the range may be limited to near the center). In the video itself, the camera filming the interaction is moving independently of the person sitting, and unless it was faked, it was able to see its own perspective.
    • nexuist 5y ago
      It seems like it does, given in the demo video there's a lady and a baby in the same shot.
      • ForHackernews 5y ago
        We don't know if the baby is seeing the full 3d effect, but my understanding of how light-field displays work is that it isn't based on eye-tracking.
  • anon_tor_12345 5y ago
    i got curious (since you said you'd seen this years ago) and found my way to your cppcon 2016 talk. good talk but in particular i want to congratulate you on the weaving pun (first multithreaded tech).
• tengbretson 5y ago
  Wouldn't there also have to be some kind of head tracking involved? Otherwise you're still limited to just showing a fixed perspective in 3d.
  • wesleyy 5y ago
    No, that's what the lightfield does. You see different physical images depending on your angle to the screen
    • defaultname 5y ago
      Fascinating. So not only is it feeding it an 8K / 30 (60?) FPS image, it's feeding it numerous incident angle variations and displaying all of them simultaneously? Sounds like a monster data rate.
      • ithkuil 5y ago
        https://news.ycombinator.com/item?id=23316225 https://news.ycombinator.com/item?id=23316225 > the 8K is their Input Resolution. > That resolution is then divided into the 45 viewing directions:
        • datameta 5y ago
          Is existing Looking Glass Factory tech the same though? Not so sure about that. Those displays are typically monitor-sized at the largest and not really aimed at displaying a live feed of a person. This looks to be a more seamless experience on a larger screen.
          • datameta 5y ago
            Downvoted, with no response, for posing an open question. Shame on you, honestly.
            • deleted 5y ago
              [deleted]
• weird-eye-issue 5y ago
  Do you think it's possible 90% of this is due to the studio quality lighting, large high quality screen, good mic/speakers, and low latency network? It seems like those factors alone would get most of the way there and the 3D aspect is just a bonus. Obviously I haven't used it in person but this was just a thought since most people are used to video calls on their small phone/laptop with poor lighting, mics, etc
  • abhv 5y ago
    NO WAY. It is impossible for me to explain how/why it works so well.
    • sneak 5y ago
      I am sure that the immersion of the experience is higher. My question (and perhaps that of GP) is: is this greater immersion actually beneficial to communication? I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing. Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical. I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
      • throwaway3699 5y ago
        Being able to feel like another person in the room is enough for me to reconsider working from home. As of right now I strongly have a preference for in person, but I do acknowledge most people prefer commute and cost benefits over productivity. The state of video conferencing today is a poor one and I'm very excited for something that can change the industry like this.
        • sneak 5y ago
          I'm right there with you, and I use a 4k camera and a boom mic and headphones and wired ethernet to videoconference now: I have been regularly complaining about the low resolution and framerates of current videoconferencing systems (10-15fps, 720p, low bitrate - and that's the highest quality setting available!). If Google wanted to make me believe they care about videoconferencing quality, they'd have a 4k 60fps option that auto-enables in Meet if it detects everyone on the call is on wired gigabit with a 4k camera.
• sleepybrett 5y ago
  Similar tech in the display of the nintendo 3ds?
• rejectedandsad 5y ago
  That's very cool, but out of curiosity > project (he has a Phd Princeton Why is this part relevant? Was the PhD in light field technology?
• soylentgraham 5y ago
  Is it lenticular, or something else?
• draw_down 5y ago
  It does seem really cool but I don't see any reason to think they are sharing it with a larger audience.
• imiric 5y ago
  It looks like they're doing photogrammetry in real time, which is mind boggling. I'm not familiar with this space, but building a 3D model, texturing it with live video, compressing and sending that over the internet, and doing it with minimal latency for it to be believable/enjoyable? Incredible technical achievement if that's the approach. Using state of the art tech, no doubt, and probably lots of ML magic to smooth the rendering. The 3D display is the icing on the cake, it must look amazing in person.
  • shahar2k 5y ago
    not that hard to do if you have actual depth sensing cameras, and even without those, something like the oculus quest 2 does that exact task (generate a rough 3d volume based on several 2d video feeds) you can see a neat example when you draw your guardian space, and move objects (and notice how it updates the 3d volume representation)
    • imiric 5y ago
      Good point, I haven't followed the latest VR advancements, that does sound neat. Still, Starline's approach is surely much more sophisticated (the hardware obviously has a lot to do with that, these are prototypes of a desk-sized machine vs a headset). The 3D model looks reasonably detailed, and the final render has very few artifacts. Making it all work over a WAN link with latencies critical for teleconferencing is also impressive.
    • Aeolun 5y ago
      Using Oculus Quest 2 had me just walking around my room in wonder about seeing straight through the headset for a bit.
    • rewq4321 5y ago
      The difficulty completely depends on the level of quality you're after. They're certainly working on cutting-edge-level quality, so it is likely no easy task. Someone else pointed out that they released a paper on related tech last year: https://augmentedperception.github.io/deepviewvideo/ https://augmentedperception.github.io/deepviewvideo/
• Pxtl 5y ago
  I'm still not understanding how the 3D works... is it like the 3DS? Because that required you position your head in a very specific place.
  • Philip-J-Fry 5y ago
    The "New 3DS" introduced head tracking and no longer needed your head in a fixed position by the way.
• speeder 5y ago
  I want to try this just to see what happens, if I will have again the feeling something is more real than reality. Because optometrists are illegal in my country (here only medics can decide what glasses you can use), currently I don't have stereoscopic vision, although my brain CAN do it, if I had the correct images sent to my eyes somehow. (one of my eyes muscles is slightly shorter than the other side thus the images on that eye are shifted unless I had an optometrist design me glasses with a prism). So when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality, despite it being obvious fantasy with aliens, floating rocks and all that stuff. EDIT: for those wondering, I am from Brazil, here medical professionals often sue the shit out of anyone offering any service remotely similar to an optometrist, they are quite aggressive about it, some attempted to make even discussion of the subject illegal. And when I was trying to get the prism and asked around my medics about it, one of them went really ballistic, I honestly thought the guy was going to punch me. I believe the reason for that is that for many medics, designing glasses is their only source of income, a guaranteed one, since here is ALSO illegal to buy glasses without a medic desining them for you first, even if the new glasses are supposed to be identical to the old ones!
  • djrogers 5y ago
    Sounds like a great reason to do a little medical tourism once the present situation is under control.
  • thegginthesky 5y ago
    Shouldn't an ophthalmologist here in Brazil be able to diagnose you and give you a prescription for corrective lenses, or even perform corrective surgery? I'm asking in case I might need to look for one in the future.
    • ZeroClickOk 5y ago
      I'm from Brazil too, and my eye glasses are recommended by a optometrist. The only way an optometrist to work is being an ophthalmologist with specialization in "optometry". It is a sad reality from our bureaucratic government that don't give us freedom to innovate.
• wpietri 5y ago
  The question for me is how much it matters after the novelty wears off. I count at least 5 waves of 3D technology starting in 1851 with the Brewster Stereoscope. Each time there's a surge of popularity driven by the legitimately amazing initial experience. And each time people slowly stop caring. People were incredibly excited about Avatar, and many thought it would change the movie industry. But how many people now go out of the way to see something in 3D?
  • suyash 5y ago
    Exactly, it's just a fancy FaceTime technology, I would be bored after few days. Tell me a problem that it solves.
    • imiric 5y ago
      > Tell me a problem that it solves. Replicating the experience of in-person communication much more closely than video and 2D displays will ever do. That's a noble research goal if nothing else, I don't get the skepticism. There are several reasons 3D content and previous generation displays didn't take off, but there's no reason to believe a revolutionary new approach and product couldn't change this (e.g. electric cars were invented in the 19th century and are only now becoming popular). AFAICT the real time photogrammetry they're using here along with the no-glasses 3D display is a major leap forward. If they can get it cheap and reliable enough to mass market, it would be a game changer. I certainly know what kind of display and teleconferencing software I want when the next pandemic hits, and it's not what we have now.
      • wpietri 5y ago
        > Replicating the experience of in-person communication much more closely That's not a problem people express much, at least not in ways where "3D" points to a solution. When I want to see people in person, it's not because of a lack of stereoscopy. I want to hug them, to break bread with them. > there's no reason to believe a revolutionary new approach and product couldn't change this There is indeed! In specific, the many times we have already had revolutionary new approaches and products that were met with great enthusiasm in the market for a few years. I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular. (I'm not sure what your work calls are like these days, but quite a lot of people turn off video in mine.) The lesson I take from that is that people mainly self-generate the feeling of interpersonal connection, and they can do it with very little in the way of cues. To me that's another strong indication that no new 3D technology will make much of a difference.
• XnoiVeX 5y ago
  Sony has it's own version of the light field display. https://youtu.be/KrLMnQM0_Ps https://youtu.be/KrLMnQM0_Ps
• mywacaday 5y ago
  I have wondered in the past if a similar result could be achieved using a 3d headset with some tracking/cameras and removing the headset from the view through a real-time deep fake that could be achieved through a short scan before the call. Would this even be feasible?
  • purplecats 5y ago
    probably would have an uncanny valley effect
• Aeolun 5y ago
  How does it do the light field if multiple people are behind the glass? They can’t really optimize it for every person individually right?
• jbverschoor 5y ago
  I've seen 3d screens in Japan 20 years ago. TVs, ad displays, mobile phones. It works really well, and it's actually very simple technology. Never understood why that never took off.
  • colordrops 5y ago
    This is different. It's a light field display, so you can view things at different angles as you move around, rather than having to sit in a very exact position and only seeing one angle.
• nipponese 5y ago
  Isn't it just the "Pepper's Ghost" technique? https://en.wikipedia.org/wiki/Pepper%27s_ghost https://en.wikipedia.org/wiki/Pepper%27s_ghost
  • nimchimpsky 5y ago
    huh ? how is it related to that in any way ?
  • rewq4321 5y ago
    No, this sends a different image to each eye. It's not actually a transparent piece of glass with an image on it[0] - I think they just try to make it seem as if it is a transparent pane you're looking through. The marketing for it is pretty terrible though, because I initially came away with the same impression as you. [0] See image here: https://www.engadget.com/google-project-starline-191228699.html https://www.engadget.com/google-project-starline-191228699.h...
• 1vuio0pswjnm7 5y ago
  Questions: How does this relate to advertising and the necessary surveillance to support it. Will this "product" be set up to phone home and "update" by default. Will the price be "free". Commercial viability. How much would someone pay for this.
  • crazygringo 5y ago
    Sheesh. Why the FUD? It would presumably work exactly the same way as Google's existing videoconferencing hardware: https://workspace.google.com/products/meet-hardware/ https://workspace.google.com/products/meet-hardware/ In other words, you pay market value for it, it doesn't include advertising, and Google is contractually obligated not to snoop on your content. Which is why even Google competitors use this hardware, because the legal and technological protections are strong enough they don't worry about Google stealing their IP.
• sfifs 5y ago
  So in my company, I've used various avatar's of Cisco TelePresence systems over many years. The big commercial need it turns out isn't so much realism as it is flexibility to accommodate people dialing in from different systems - phones, laptops, different types of telepresence setups from small single room to big conference rooms or even telephone connections etc. Many years ago, we were all wowed by the life-size realism and had people come into offices for it . Nowadays these meetings have lots of people crammed in across multiple screens dialing in as they please and all the better for everyone :-)
• supermatt 5y ago
  Over a decade ago I used a holographic video conferencing system that used kinect for depth mapping, it was very convincing. It would also reorient the display based on your head position. so you could look "around" the avatar. There are demos of binocular 3d conferencing done with a lenticular display (lookingglass), although those large displays are extortionately priced (1/4 of the size of this google one is $3000...) - keeping them out of the hands of most devs. No doubt google are doing the same, but can afford these larger displays. You can easily find examples (and research papers) by googling for the relevant terms. Google claiming they have invented a "new technology" just shits on all those folk who dont have the publicity/funding of google.
• estaseuropano 5y ago
  Just very practically, is this a $xxxx.xx improvement over simple video chat with a large screen and simple camera? Will this even work without low-latency high-bandwidth connection up and down? It seems more like an art project than a tech usable in the coming decade.
• niutech 5y ago
  Can you tell your friend at Google that their website weights whooping 89 MB, most of which are huge animated GIF-s? This is insane! Doesn't Google know about HTML5 video yet? https://www.smashingmagazine.com/2018/11/gif-to-video/ https://www.smashingmagazine.com/2018/11/gif-to-video/