9 ms·

An engineer replaced four monitors with Meta glasses

84 points by stesch 11y ago

Kenji 11y ago
I remain sceptical about how ergonomic this is. I also wonder how much processing power they need to run this in real-time. Such virtual reality simulations are often still somewhat computationally expensive and sophisticated. But it actually looks quite exciting.
chimitos 11y ago
I'm a big fan of his making this in Unity
mgberlin 11y ago
For me, a non-gamer, this is the killer app for any of the new head-mounted displays. I've been waiting for years for some sort of goggles I can plug into my dev machine, put on to block out all visual distractions, and get work done.
- aNoob7000 11y ago
  I've been waiting for the same thing. I'd also like to have N virtual monitors that I could either switch through or see all at once (like a big wall of screens).
- bitwize 11y ago
  I think strapping gear onto our heads and flying around the Gibson like Zero Cool has been a dream of many of us for a long time. Too bad the actual ergonomics of such displays, even for small trivial tasks like "get this part, it's on your left, now get this part, it's on your right and just above your head", are still shit, making it an out-of-reach fantasy.
- SapphireSun 11y ago
  Wait until you can start hanging with your friends when you're halfway around the world. The HTC Vive is amazing in positional fidelity and in pixel density. I tried it out.
- reustle 11y ago
  Putting on goggles to block out all visual distractions? An office with a door does this pretty well too.
  - mgberlin 11y ago
    And yet that's somehow harder to find than a high resolution, ergonomic, HMD in the Bay Area.
TrevorJ 11y ago
Article is really thin on details, I'm skeptical. Last I checked, pixel density was still a big issue with HMD's and text. Contrast, as well is a problem with AR style HMD's. I'd love it if the tech really was there though, it would be awesome. I actually think using some flavor of DLP tech and a foveated display with eye tracking might allow for phenomenal pixel density, wonder if anyone is working on that yet.
- mappu 11y ago
  SMI are working on fundamental technology for foveated displays with eye tracking. There's also an HMD from FOVE on Kickstarter that intends to ship to backers in "Fall 2016". John Carmack suggested on Twitter that foveated rendering won't be a performance win until a few more generations, when higher-resolution panels are more commonplace. Right now the necessary tracking and layering could actually reduce performance compared to rendering the full scene.
  - knughit 11y ago
    There are applications where high resolution is interesting but high frequency is not.
    - mappu 11y ago
      High refresh rate is mandatory to reduce nausea.
      - wlesieutre 11y ago
        Additionally, every time you move your eyes, the high-detail area has to shift with them. If you did foveated rendering without low latency and high refresh rate, you're going to get visible pop-in of render detail levels. Haven't tried it, but I'd hazard a guess that it would be really uncomfortable.
- dshankar 11y ago
  There is no such thing as a `foveated display` There is foveated rendering: a rendering optimization technique where you render at higher resolutions where the eye is looking at due to higher visual acuity in a small ~5-degree circle thanks to a large concentration of cones in the fovea of the eye. This has nothing to do with the actual pixel density of HMDs. Physical pixel density remains an issue, and foveated rendering does not solve that. edit: Perhaps I misunderstood, are you talking about retinal laser projected displays and concentrating the beam on the fovea?
MollyR 11y ago
Wow if I could use this for coding at work, instead of the using the bad monitors they have. That would be amazing ! Does anyone know more about this tech?
- 8611m 11y ago
  This article is probably referring to Robert Scoble's live stream from Meta's office, you can see the whole video here. There's an engineer who claims to have replaced his multi-monitor setup with an AR device by Meta demoed at TED. The TED video will be released this Wednesday. Video stream link here: https://www.facebook.com/RobertScoble/videos/10153928655124655/?pnref=story https://www.facebook.com/RobertScoble/videos/101539286551246... Edited for link.
mappu 11y ago
I've used http://vrdesktop.net/ http://vrdesktop.net/ with the Oculus DK2, but, the resolution was still a little low for getting real work done. I'm skeptical that this HMD would have better resolution/tracking than Oculus. I'm hopeful the CV1 will make it at least bearable, because there are so many possible benefits (custom environments, isolating distractions, privacy, ...) You could add multiple virtual monitors as long as Windows believes they exist (there is http://virtualmonitor.github.io/ http://virtualmonitor.github.io/ for XPDM, but i'm not sure how well that would work nowadays).
- imaginenore 11y ago
  Nope, the resolution is still way too low, the FOV is too big. We need something like 4K per eye for it to become comfortable. And even that won't even get close to 3 physical monitors in terms of resolution, simple because the geometric transforms kill your perfect pixels.
  - mappu 11y ago
    >And even that won't even get close to 3 physical monitors in terms of resolution, simple because the geometric transforms kill your perfect pixels. I have 3 physical monitors at $DAYJOB and i don't think i see 1:1 pixels on all three simultaneously - certainly i do for the monitor i'm focusing on, but less so on the others, or maybe only peripheral changes. So i think the HMD could get by with a slightly reduced resolution as long as the FOV accommodated peripheral vision.
iamleppert 11y ago
This is one of those things you do because it sounds cool and then you go back to getting actual work done with your real world monitors.
- dexterdog 11y ago
  For now, but this is the future of HD computer displays.
platz 11y ago
projecting a flat concept onto a 3d world seems like a category mismatch. Turning your head is not going to be faster than Meta-Tabbing to another virtual desktop.
- monsieurbanana 11y ago
  The 3d world doesn't aim to replace the virtual desktops, it aims to replace the multi-monitors.
  - platz 11y ago
    I can think of a few professions that need multi at the same time, but since yoy can only interact with one at a time, most probably wouldnt be helped by this, including programmers.
    - Stratoscope 11y ago
      Of course you can interact with two monitors at once. I do it every day. My current standing desk setup is an MBPR15 on top of a wine case on my desk, with a ViewSonic 24" 4K display to the left, hanging in portrait mode on the Amazon Basics version of an Ergotron arm. Best dev rig I've ever had! Quite often I'll put docs and references on the external monitor while I write code in Visual Studio or IntelliJ on the Mac's display. This works especially well with the monitor in portrait mode. Then I leave the mouse cursor over on the external display and use two-finger scrolling there, while using the keyboard in the editor window. If this isn't interacting with two monitors at once, I don't know what is. And it's a huge improvement over a single monitor.
      - XorNot 11y ago
        I'm interacting with 3 monitors right now (well, was, then I got distracted and started reading HN). I think the more important way to unpack that assertion though, is that if you watch people studying using pens and paper, almost everyone has a specific way they end up laying out papers and books on their desk - and the reason we still do it that way, by and large, is because you get a lot more desk real estate then you get screen real estate.
deleted 11y ago
[deleted]
al2o3cr 11y ago
Anybody seen specs for the thing they're demoing in this? The Kickstarter (https://www.kickstarter.com/projects/551975293/meta-the-most-advanced-augmented-reality-interface/description https://www.kickstarter.com/projects/551975293/meta-the-most...) claims 960x540 displays in each eye, which would make for a jaggy-tastic text rendering experience...
- semiel 11y ago
  That was the previous version of the glasses. These are supposed to be much better, but we won't see specs until at least Wednesday, when the press embargo lifts.
- michaelbuckbee 11y ago
  That was for the Meta1 (which they sold out of).
vernie 11y ago
Sure he did.
bitwize 11y ago
Not interested. Not until they fix the eyehurt problem. AR glasses have this problem wherein they don't detect where your eyes are or should be focusing. So they throw an image up at some default focal depth, and if that image happens to overlay some of my surroundings at a different focal depth, then my eyes' focusing muscles start "hunting" and that can lead to significant eyestrain. I experienced this while trying out some state-of-the-art Sony goggles one of their sales reps was demoing at my workplace.
- josephpmay 11y ago
  It's called the vergence-accommodation conflict. And I've solved it (you'll have to wait awhile to hear about that). But it's not actually as big of a deal as it sounds, at least for most people. Crosstalk is a more common source of eyestrain and luckily easy to fix.
  - AndrewKemendo 11y ago
    >And I've solved it (you'll have to wait awhile to hear about that). Why tease it like that? Why not just say how you solved it? There are a handful of people who have published options [1] and nobody has ripped them off. In fact we have a whitepaper that does the same and I'm happy to share our approach which is also a VRD system based on pinlight display with some persistence of vision tricks. Quick, everyone sign an NDA! [1] https://www.youtube.com/watch?v=P407DFm0PFQ https://www.youtube.com/watch?v=P407DFm0PFQ
    - josephpmay 11y ago
      You're right. It was an asshole thing for me to say, and I apologize. We haven't filed a patent yet, and my team has agreed on the advise of a lawyer that it would be best not to disclose the technical details. The principal on how it works is very similar to Hong Hua's fused depth plane technique[0], except we've found a way to do it that's compact and light enough to put in a headset. I've used that pinlight display prototype before, and it's one of the best AR displays I've tried. I'm really interested in what your team is working on. Is it something you plan on commercializing? [0] https://www.osapublishing.org/jdt/abstract.cfm?uri=jdt-10-4-308 https://www.osapublishing.org/jdt/abstract.cfm?uri=jdt-10-4-...
angrydev 11y ago
The title should read 'A Meta engineer replaces four monitors with Meta glasses'
deleted 11y ago
[deleted]
henryslater 11y ago
It seems weird that he didn't actually log into spotify.
milesf 11y ago
Huh. I didn't know Christian Slater owned a tech company :P http://i.imgur.com/MREkcju.jpg http://i.imgur.com/MREkcju.jpg
melling 11y ago
"The keyboard is the only thing I need, except for the glasses." I think this loses a lot of appeal until we can throw away the keyboard. We should be able to do better. Hopefully, Google ships this silicon this year: https://www.youtube.com/watch?v=0QNiZfSsPc0 https://www.youtube.com/watch?v=0QNiZfSsPc0 I'd rather have those monitors and no keyboard.
- Animats 11y ago
  Now that's a step forward. Gloves in VR without force feedback have sucked. The insight here is that there are new kinds of gestures, such as touching fingers together and rubbing them sideways. They're able to sense those gestures with Doppler radar. They probably don't even try to resolve the gestures into positions; they may just throw a machine learning algorithm at the problem of recognizing the gestures from the Doppler radar outputs. This starts to make VR look useful for something other than roller coaster simulators. Google has a nice demo of a volume control knob. Will this scale up to a virtual mix board?
michaelbuckbee 11y ago
There is another teaser video at http://www.metavision.com http://www.metavision.com
marcus_holmes 11y ago
Oculus Vim!
lololomg 11y ago
On my X Box I don't care if the image resolution is low cause it doesn't matter that much for games or video. But for work I need lots of pixels. It's going to be a few years before that will fit in VR goggles.
mchahn 11y ago
If I understand this correctly you can also see the real world because the image is overlayed. This would be too distracting for me. The opposite, a pair of glasses that totally blacked out the rest of the world, would be ideal for me. Put on some sound-cancelling headphones and I'm in programming heaven. Related: Some editor users set up their background color as partially transparent and see other browser windows, etc., behind the text they are editing. This would drive me crazy, not to mention that my eyes are bad enough to begin with.

84 points by stesch 11y ago

Kenji 11y ago

I remain sceptical about how ergonomic this is. I also wonder how much processing power they need to run this in real-time. Such virtual reality simulations are often still somewhat computationally expensive and sophisticated. But it actually looks quite exciting.

chimitos 11y ago

I'm a big fan of his making this in Unity

mgberlin 11y ago

For me, a non-gamer, this is the killer app for any of the new head-mounted displays. I've been waiting for years for some sort of goggles I can plug into my dev machine, put on to block out all visual distractions, and get work done.

aNoob7000 11y ago
I've been waiting for the same thing. I'd also like to have N virtual monitors that I could either switch through or see all at once (like a big wall of screens).
bitwize 11y ago
I think strapping gear onto our heads and flying around the Gibson like Zero Cool has been a dream of many of us for a long time. Too bad the actual ergonomics of such displays, even for small trivial tasks like "get this part, it's on your left, now get this part, it's on your right and just above your head", are still shit, making it an out-of-reach fantasy.
SapphireSun 11y ago
Wait until you can start hanging with your friends when you're halfway around the world. The HTC Vive is amazing in positional fidelity and in pixel density. I tried it out.
reustle 11y ago
Putting on goggles to block out all visual distractions? An office with a door does this pretty well too.
- mgberlin 11y ago
  And yet that's somehow harder to find than a high resolution, ergonomic, HMD in the Bay Area.

TrevorJ 11y ago

Article is really thin on details, I'm skeptical. Last I checked, pixel density was still a big issue with HMD's and text. Contrast, as well is a problem with AR style HMD's. I'd love it if the tech really was there though, it would be awesome. I actually think using some flavor of DLP tech and a foveated display with eye tracking might allow for phenomenal pixel density, wonder if anyone is working on that yet.

mappu 11y ago
SMI are working on fundamental technology for foveated displays with eye tracking. There's also an HMD from FOVE on Kickstarter that intends to ship to backers in "Fall 2016". John Carmack suggested on Twitter that foveated rendering won't be a performance win until a few more generations, when higher-resolution panels are more commonplace. Right now the necessary tracking and layering could actually reduce performance compared to rendering the full scene.
- knughit 11y ago
  There are applications where high resolution is interesting but high frequency is not.
  - mappu 11y ago
    High refresh rate is mandatory to reduce nausea.
    - wlesieutre 11y ago
      Additionally, every time you move your eyes, the high-detail area has to shift with them. If you did foveated rendering without low latency and high refresh rate, you're going to get visible pop-in of render detail levels. Haven't tried it, but I'd hazard a guess that it would be really uncomfortable.
dshankar 11y ago
There is no such thing as a `foveated display` There is foveated rendering: a rendering optimization technique where you render at higher resolutions where the eye is looking at due to higher visual acuity in a small ~5-degree circle thanks to a large concentration of cones in the fovea of the eye. This has nothing to do with the actual pixel density of HMDs. Physical pixel density remains an issue, and foveated rendering does not solve that. edit: Perhaps I misunderstood, are you talking about retinal laser projected displays and concentrating the beam on the fovea?

MollyR 11y ago

Wow if I could use this for coding at work, instead of the using the bad monitors they have. That would be amazing ! Does anyone know more about this tech?

8611m 11y ago
This article is probably referring to Robert Scoble's live stream from Meta's office, you can see the whole video here. There's an engineer who claims to have replaced his multi-monitor setup with an AR device by Meta demoed at TED. The TED video will be released this Wednesday. Video stream link here: https://www.facebook.com/RobertScoble/videos/10153928655124655/?pnref=story https://www.facebook.com/RobertScoble/videos/101539286551246... Edited for link.

mappu 11y ago

I've used http://vrdesktop.net/ http://vrdesktop.net/ with the Oculus DK2, but, the resolution was still a little low for getting real work done. I'm skeptical that this HMD would have better resolution/tracking than Oculus. I'm hopeful the CV1 will make it at least bearable, because there are so many possible benefits (custom environments, isolating distractions, privacy, ...) You could add multiple virtual monitors as long as Windows believes they exist (there is http://virtualmonitor.github.io/ http://virtualmonitor.github.io/ for XPDM, but i'm not sure how well that would work nowadays).

imaginenore 11y ago
Nope, the resolution is still way too low, the FOV is too big. We need something like 4K per eye for it to become comfortable. And even that won't even get close to 3 physical monitors in terms of resolution, simple because the geometric transforms kill your perfect pixels.
- mappu 11y ago
  >And even that won't even get close to 3 physical monitors in terms of resolution, simple because the geometric transforms kill your perfect pixels. I have 3 physical monitors at $DAYJOB and i don't think i see 1:1 pixels on all three simultaneously - certainly i do for the monitor i'm focusing on, but less so on the others, or maybe only peripheral changes. So i think the HMD could get by with a slightly reduced resolution as long as the FOV accommodated peripheral vision.

iamleppert 11y ago

This is one of those things you do because it sounds cool and then you go back to getting actual work done with your real world monitors.

dexterdog 11y ago
For now, but this is the future of HD computer displays.

platz 11y ago

projecting a flat concept onto a 3d world seems like a category mismatch. Turning your head is not going to be faster than Meta-Tabbing to another virtual desktop.

monsieurbanana 11y ago
The 3d world doesn't aim to replace the virtual desktops, it aims to replace the multi-monitors.
- platz 11y ago
  I can think of a few professions that need multi at the same time, but since yoy can only interact with one at a time, most probably wouldnt be helped by this, including programmers.
  - Stratoscope 11y ago
    Of course you can interact with two monitors at once. I do it every day. My current standing desk setup is an MBPR15 on top of a wine case on my desk, with a ViewSonic 24" 4K display to the left, hanging in portrait mode on the Amazon Basics version of an Ergotron arm. Best dev rig I've ever had! Quite often I'll put docs and references on the external monitor while I write code in Visual Studio or IntelliJ on the Mac's display. This works especially well with the monitor in portrait mode. Then I leave the mouse cursor over on the external display and use two-finger scrolling there, while using the keyboard in the editor window. If this isn't interacting with two monitors at once, I don't know what is. And it's a huge improvement over a single monitor.
    - XorNot 11y ago
      I'm interacting with 3 monitors right now (well, was, then I got distracted and started reading HN). I think the more important way to unpack that assertion though, is that if you watch people studying using pens and paper, almost everyone has a specific way they end up laying out papers and books on their desk - and the reason we still do it that way, by and large, is because you get a lot more desk real estate then you get screen real estate.

deleted 11y ago

[deleted]

al2o3cr 11y ago

Anybody seen specs for the thing they're demoing in this? The Kickstarter (https://www.kickstarter.com/projects/551975293/meta-the-most-advanced-augmented-reality-interface/description https://www.kickstarter.com/projects/551975293/meta-the-most...) claims 960x540 displays in each eye, which would make for a jaggy-tastic text rendering experience...

semiel 11y ago
That was the previous version of the glasses. These are supposed to be much better, but we won't see specs until at least Wednesday, when the press embargo lifts.
michaelbuckbee 11y ago
That was for the Meta1 (which they sold out of).

vernie 11y ago

Sure he did.

bitwize 11y ago

Not interested. Not until they fix the eyehurt problem. AR glasses have this problem wherein they don't detect where your eyes are or should be focusing. So they throw an image up at some default focal depth, and if that image happens to overlay some of my surroundings at a different focal depth, then my eyes' focusing muscles start "hunting" and that can lead to significant eyestrain. I experienced this while trying out some state-of-the-art Sony goggles one of their sales reps was demoing at my workplace.

josephpmay 11y ago
It's called the vergence-accommodation conflict. And I've solved it (you'll have to wait awhile to hear about that). But it's not actually as big of a deal as it sounds, at least for most people. Crosstalk is a more common source of eyestrain and luckily easy to fix.
- AndrewKemendo 11y ago
  >And I've solved it (you'll have to wait awhile to hear about that). Why tease it like that? Why not just say how you solved it? There are a handful of people who have published options [1] and nobody has ripped them off. In fact we have a whitepaper that does the same and I'm happy to share our approach which is also a VRD system based on pinlight display with some persistence of vision tricks. Quick, everyone sign an NDA! [1] https://www.youtube.com/watch?v=P407DFm0PFQ https://www.youtube.com/watch?v=P407DFm0PFQ
  - josephpmay 11y ago
    You're right. It was an asshole thing for me to say, and I apologize. We haven't filed a patent yet, and my team has agreed on the advise of a lawyer that it would be best not to disclose the technical details. The principal on how it works is very similar to Hong Hua's fused depth plane technique[0], except we've found a way to do it that's compact and light enough to put in a headset. I've used that pinlight display prototype before, and it's one of the best AR displays I've tried. I'm really interested in what your team is working on. Is it something you plan on commercializing? [0] https://www.osapublishing.org/jdt/abstract.cfm?uri=jdt-10-4-308 https://www.osapublishing.org/jdt/abstract.cfm?uri=jdt-10-4-...

angrydev 11y ago

The title should read 'A Meta engineer replaces four monitors with Meta glasses'

deleted 11y ago

[deleted]

henryslater 11y ago

It seems weird that he didn't actually log into spotify.

milesf 11y ago

Huh. I didn't know Christian Slater owned a tech company :P http://i.imgur.com/MREkcju.jpg http://i.imgur.com/MREkcju.jpg

melling 11y ago

"The keyboard is the only thing I need, except for the glasses." I think this loses a lot of appeal until we can throw away the keyboard. We should be able to do better. Hopefully, Google ships this silicon this year: https://www.youtube.com/watch?v=0QNiZfSsPc0 https://www.youtube.com/watch?v=0QNiZfSsPc0 I'd rather have those monitors and no keyboard.

Animats 11y ago
Now that's a step forward. Gloves in VR without force feedback have sucked. The insight here is that there are new kinds of gestures, such as touching fingers together and rubbing them sideways. They're able to sense those gestures with Doppler radar. They probably don't even try to resolve the gestures into positions; they may just throw a machine learning algorithm at the problem of recognizing the gestures from the Doppler radar outputs. This starts to make VR look useful for something other than roller coaster simulators. Google has a nice demo of a volume control knob. Will this scale up to a virtual mix board?

michaelbuckbee 11y ago

There is another teaser video at http://www.metavision.com http://www.metavision.com

marcus_holmes 11y ago

Oculus Vim!

lololomg 11y ago

On my X Box I don't care if the image resolution is low cause it doesn't matter that much for games or video. But for work I need lots of pixels. It's going to be a few years before that will fit in VR goggles.

mchahn 11y ago

If I understand this correctly you can also see the real world because the image is overlayed. This would be too distracting for me. The opposite, a pair of glasses that totally blacked out the rest of the world, would be ideal for me. Put on some sound-cancelling headphones and I'm in programming heaven. Related: Some editor users set up their background color as partially transparent and see other browser windows, etc., behind the text they are editing. This would drive me crazy, not to mention that my eyes are bad enough to begin with.