12 ms·

Microsoft’s HoloLens 2: a $3,500 mixed-reality headset for the factory

285 points by T-A 8y ago

mrlaserOO 8y ago
That’s pretty exciting, especially the doubled field of view
serioussecurity 8y ago
Also selling it to the military to "increase lethality". https://www.theverge.com/2019/2/22/18236116/microsoft-hololens-army-contract-workers-letter https://www.theverge.com/2019/2/22/18236116/microsoft-holole...
- ABeeSea 8y ago
  Well I assume the military generally doesn’t want decreased lethality when purchasing equipment.
  - setquk 8y ago
    Actually they do. Only because collateral is a PR nightmare.
    - derp_dee_derp 8y ago
      No they don't. Collateral damage is a function of inefficient lethality (killing people who are not your target is wasteful and inefficient). Increasing lethality decreases collateral.
      - setquk 8y ago
        No that's not quite right. Lethality is the effectiveness of desired outcome. Was the target eliminated? Collateral is the side effects of the desired outcome. Was anyone or anything else eliminated? They are two different metrics. Consider two contrasting analogies which break your understanding: Ballistic missile: high lethality, high collateral probability. Polonium tea: high lethality, low collateral probability. About 20 years ago, I was working in the defence sector, on guidance systems, as an engineer. Sat through many a powerpoint about how collateral could be marketed away. Turns out they won that battle eventually... Incidentally I bailed out when I saw what a fucking mess of a world I was creating.
        derp_dee_derp 8y ago
        This is bad logic. By your metrics, the most lethal thing in the world is a rusty spoon stabbed into some ones eye; either it kills the target instantly or kills them later due to infection. Very close to 100% death rate. Very lethal. No collateral damages, either. And yet no professional militaries are armed with rusty spoons
        Dylan16807 8y ago
        I don't know what logic you're seeing but that's not an accurate description. A spoon is low lethality, low collateral.
        dragonsngoblins 8y ago
        That's because the lethality of a rusty spoon is actually very low... The lethality of having been stabbed in the eye might be high, but not of spoons themselves since it is hard to successfully wound the eye of an enemy soldier with a rusty spoon at a distance when said enemy soldier is armed with ranged weaponry, and it is still difficult to do even in melee range relative to how easy it is to wound with bayonets and combat knives.
        setquk 8y ago
        They had swords that did that. Then they found out that guns had higher lethality with lower risk to the attacker (another primary metric). Collateral wasn't much of a problem back then. No one gave a shit.
        imtringued 8y ago
        What is a bullet if not a very carefully shaped rusty spoon stabbed into someone via a complicated launching mechanism?
    - 420codebro 8y ago
      I know I'm splitting hairs, but they want lethal and accurate weapons. When they pull the trigger, they want the person they intend to "neutralize" to only be hit. They do not want to decrease their lethality. The threat of usage of a weapon (lethality) is what keeps a lot of engagements very small, or not even happening at all.
- writepub 8y ago
  The military also buys iPhones, and Androids. What phone do you use? That very phone was probably used to order a drone strike! Did you use GPS technology this past week? That was the tech guiding the drone to it's target. And this thing called the internet was invented to network the military
  - roywiggins 8y ago
    GPS is a military technology opened to civilians... going the other way feels different, for whatever reason.
    - nradov 8y ago
      No, GPS was actually designed from the start as a dual military and civilian system. The only part that wasn't originally available to civilians was the high precision signal.
      - sdinsn 8y ago
        > GPS was actually designed from the start as a dual military and civilian system No, what prompted GPS to be opened to civilian uses was KAL007 being shot down by the Soviet Union. [1] 269 killed, including a US Rep. [1] https://en.wikipedia.org/wiki/Korean_Air_Lines_Flight_007 https://en.wikipedia.org/wiki/Korean_Air_Lines_Flight_007
        nradov 8y ago
        No, as usual Wikipedia isn't quite accurate. Here's an actual interview with the "father of GPS". "BRAD PARKINSON: Absolutely. And there are a lot of misconceptions about that. You hear people saying, well it was a military system and eventually made a civil system. That was not true. When I testified before Congress back about 1975, I said this is a combination military civil system." https://www.sciencefriday.com/segments/how-gps-found-its-way/ https://www.sciencefriday.com/segments/how-gps-found-its-way...
  - 420codebro 8y ago
- int_19h 8y ago
  I really don't get the opposition to this. Either we need the military, or we don't. If we do, then we want to have one that's can beat anything that could be thrown at it with as few casualties as possible. If we don't, then going after VR devices rather than the rifles is all wrong on priorities.
  - s3r3nity 8y ago
    Agreed - to pretend that other countries aren't devoting the top tech talent to advance their capabilities in a similar way is beyond naive. I _want_ to agree with this "ethical opposition" approach, but I know too much game theory / history to know that I also don't want my power grid knocked out on a moment's notice because, say, Googlers opposed stronger cyber security for our military.
    - int_19h 8y ago
      I can think of specific projects that might be unethical in that sense, and cannot be justified even by "well, they're doing that". But I can't see why this one would be in that category.
yodon 8y ago
2x field of view (or possibly area of view, description could be read either way). Eye tracking. MEMS display. High grade low latency hand tracking. Improved set of default gesture recognizers. Spatial Anchors service for cross platform localizing (see same AR in same location on HoloLens, ARKit, AR Core). Some sort of geometry streaming service. Support for hardware customization (hardhats, etc). Also showed a separate stand-alone IoT Kinect + CPU/GPU hardware device.
dchichkov 8y ago
Nice. AR for the industry is an interesting field. I've worked a bit on real-time recognition and registration of CAD to point cloud, for another industrial grade AR headset, developed by another company. But unfortunately an exec from Qualcomm came in, brought in his buddies and flown the thing into the ground. Hopefully Microsoft will pick up the flag of AR for industrial use.
- Barrin92 8y ago
  >Nice. AR for the industry is an interesting field Yep I'm also happy about this and like the direction Microsoft seems to be going. Industrial applications have always seemed much more interesting and potentially widely beneficial than just targeting consumers and entertainment.
Klathmon 8y ago
So does anyone think this is at the point where I can pick one up and do away with multiple monitors? (Hypothetically, I see they aren't selling to consumers yet) I play a game called Elite Dangerous with my Oculus rift, and the UX of being able to have holographic windows around my person that I can look at to activate and control is so insanely powerful and so freeing, and I'm literally willing to drop multiple thousands of dollars on something right now that can give that experience with normal desktop software right now. Sadly the Rift lacks any way to see the real world when it's on (which makes using a keyboard and mouse more difficult), and the resolution still has a ways to go before I can comfortably use it like this, not to mention the software still needs work for this use case. But I can't wait until the day when I can get rid of the monitors on my desk and replace them with a headset. Being able to place normal 2d windows around my person, be able to maybe use some limited gestures to control them, and then the same keyboard and potentially mouse that in used to.
- thefounder 8y ago
  Both Hololens and Oculus are far from being replacements for traditional displays(i.e for programming)
  - Klathmon 8y ago
    Yes, but why? What is the limiting factor here, because from my perspective it's only missing a few small things on each platform that would solve it. I'd be honestly happy with my Rift as-is if they bumped up the resolution a bit, and had some way to see the real world with the headset on (even camera passthrough is fine!). I've used virtual desktop apps in the Rift, and they aren't bad to use, and playing games like tabletop simulator is so intuitive and nice that I'm floored tht this hasn't been tried commercially yet. I assume there are some pretty significant hurtles that I'm missing, or maybe the market is still hilariously small at this point, but it just seems so close!
    - poutine 8y ago
      The resolution is way too low. Needs to be 4-16x greater for desktop replacement. Also the headsets are too hot and bulky to comfortably wear for a full workday.
Animats 8y ago
It's not a hologram, people. It's a very clever projector, but it's not a hologram at all. The example given, showing someone where to put a bolt, is about making a human do a robot's job. The computer decides what has to be done, and tells the human exactly what to do. Like the picking system in Amazon warehouses. We're getting ever closer to Marshall Brain's "Manna". Machines should think. People should work.
lwansbrough 8y ago
The piano demo was by far the most exciting for me as someone who developed for the first HoloLens. Having used the first one extensively, the improved gestures support is IMO a more significant improvement than the FOV upgrade for long term use cases. I’m interested to see how Firefox handles web browsing on HoloLens. I’ve always envisioned URLs in that context as a way of loading a new holographic experience into a shared space, as opposed to each URL providing its own separate experience (though this would be good as an option.)
roywiggins 8y ago
I played with a HoloLens 1 briefly, which was both a revelation- AR feels like it could be useful in a way VR doesn't- and very frustrating. The worst part was the gesture recognition, which was the absolute bare minimum. The Air Tap was horrible. Twice the FoV and with real hand tracking? That suddenly makes this thing viable in one step.
- Groxx 8y ago
  I generally feel like anything that requires your hands to be in the field of view (at anything near its current size) is an immediate non-starter, which is part of what ruined Air Tap for me. Plus it was just horribly imprecise. We do stuff with our hands largely away from the center of our vision. You might make a brief glance to calibrate yourself, but e.g. grabbing a thing doesn't require constant focus on the thing through the entire act.
  - ocdtrekkie 8y ago
    This may be the least insurmountable challenge though: A pair of smart gloves will mostly solve this problem.
    - Groxx 8y ago
      yep, that'd be the most comprehensive way certainly. downward-facing cameras could probably get most of it tho.
  - freeone3000 8y ago
    Air tap doesn't actually require your finger to be in field of view. Even in the Hololens 1, there is, in fact, a downward-facing camera for gestures whose field-of-camera extends all the way down to your waist.
lwansbrough 8y ago
I was disappointed to see Microsoft is still working with the concept of apps. I think we need to rethink how we start experiences in mixed reality. Pointing and tapping an app makes sense for a touch/mouse device, but not for a device that's supposed to continuously immerse the user. Here's my alternative concept: We need a search engine for holographic layer services. It would be like Google, but for MR experiences. Holographic services would use a protocol that defines a geofence to be discovered by the layer search engine's crawler over the internet (this could just be a meta tag on a classic website). The HoloLens or whatever MR device would continuously ping the search engine with its location, and the results would be ordered based on their relevance (size of geofence and proximity are good indicators). The MR device would then show the most relevant available layer in the corner of the FOV. Selecting the layer would allow enabling it either once or always, and the device would then deliver the holographic layer over the internet. The holographic layer would behave like a web service worker (in fact, it could be a web service worker) and would augment a shared experience which contains other active holographic layers. For example, your Google Maps holographic layer could be providing you with a path to walk to the nearest Starbucks, and once you're outside Starbucks, the Starbucks layer is also activated, which allows you to place an order. This concept of activated layers, I think, is a great way to avoid a future where we're being bombarded with augmented signage and unwanted experiences. In fact, you could go further and enable blocking notifications about specific/certain types of available services. (ie. don't notify me about bars or fast food restaurants.)
- janoc 8y ago
  That would make zero sense for Hololens - Hololens is an enterprise device that will in practice run one or two applications (e.g. the company training or maintenance tool). Also most industrial networks are behind heavy firewalls, so anything that actually depends on internet access is a problem. Hololens also doesn't really work outdoors (display contrast and tracking are the largest limitations), so there is no point of designing a system for supporting that when the hardware is not usable in such scenario.
  - lwansbrough 8y ago
    It would make zero sense for HoloLens today. The HoloLens doesn't even have a GPS chip. But Microsoft is building the future of MR in general. It's a space they want to own or at least lead, and their investment in Azure services for HoloLens today are proof of that. Obviously what I've described does not really favour industrial use (though I dispute the idea that this would not be achievable in industrial/enterprise environments.) Indeed, HoloLens isn't great outdoors, however the current technology is capable of it (despite the HL design.) The visor can be darkened to function more like sunglasses, which solves the contrast problem. I'm just looking 5-10 years out, in terms of what's going to be needed.
loosetypes 8y ago
So frustrating when videos I think of as marketing on YouTube are preceded by other ads. I get that this content is from theverge, not Microsoft, but the same could be said for YT shoving ads at you when you're already just trying to watch a movie trailer. I'm sure a lot of folks rarely watch many entire segments of ads unless it's on their terms. Does further ad-walling that content not just reduce the overall number of eyeballs? It's like the only ad metrics seen as meaningful are the hostile ones..
- deleted 8y ago
  [deleted]
- jobigoud 8y ago
  My other pet peeve is movie trailers channels that start their videos with a 2-second mini trailer of the trailer.
daenz 8y ago
What's cool about the use-case they describe is that it's the perfect bridge between what humans are good at: improvised problem solving and movement, and what computers are good at: crunching and surfacing data. Is this the sweet spot?
- bob_theslob646 8y ago
  >Is this the sweet spot? Yes it is! What is actually hilarious about this tech is that as the hardware gets better, the information it feeds you will ultimately be the most valuable piece. Crazy to think about when verifying validity a firehouse amount of data.
___karim 8y ago
It’s a fascinating space. I’m Waiting for a startup, the ‘Apple’ of mixed reality, to take a fresh approach and create something for the rest of us.
mtgx 8y ago
For the war
kjaer 8y ago
For finger tracking, version 1 used random forests [1], because of the performance/hardware budget trade-off: they're harder to train than a traditional deep learning algorithm, but are much more efficient to compute on the device (branching being basically free on a CPU). Version 2 uses a deep learning accelerator [2], which makes it possible to do the heavier computation of DNNs (which involve floating-point operations, which would be much more expensive on the CPU). From an engineering perspective, I just love seeing how it touches all abstraction layers of the stack, and the types of solutions that come out of thinking about the silicon and the high-level ML models at the same time. [1] https://www.microsoft.com/en-us/research/wp-content/uploads/2017/05/Towards-realistic-hands-gesture-interface.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/... [2] https://homes.cs.washington.edu/~kklebeck/lebeck-tech18.pdf https://homes.cs.washington.edu/~kklebeck/lebeck-tech18.pdf
- kevingadd 8y ago
  Unless the paper specifically calls it out in a spot I didn't see, it's not necessarily the case that the DNN operations are floating-point. Some networks use FP16 or FP32 (it's my understanding that this is very common during training) but actual production use of a trained network can happen using int8 or int4. You can see this if you look at what the 'Tensor' cores in modern geforce cards expose support for and what Google's latest cloud tensor cores support. NV's latest cores expose small matrices of FP16, INT8 and INT4 (I've seen some suggestions that they do FP32 as well but it's not clear whether this is accurate), while Google's expose huge matrices in different formats (TPUv1 was apparently INT8, TPUv2 appears to be a mix of FP16 and FP32). In non-DNN image processing it's quite common to use ints as well (iDCT, FFT, etc) for the potential performance gains vs. floating point.
- munib_ca 8y ago
  When you mention version 1 and 2, are you referring to the original hololens and the new one?
  - kjaer 8y ago
    Yes. I'm referring to the original HoloLens and HoloLens 2, so to the hardware versions.
telltruth 8y ago
This is quite disappointing. Lot of good people I've known who joined this team at early stage are all gone and it shows. Kipman simply didn't seem to get this. Magic Leap should have at least taught them that it is very bad idea to have so much weight on headset. The AR is pretty much dead at this point and all these push towards creating demos for factories is pure media stunt. They even managed to lure US army and I'd like to know the story of how they did this but from my own experience trying to use Hololens in such situations would be a tax, not benefit. The battery life is miserable, its too heavy for wearing on head and induces headaches for most people. Also, using it in custom situation requires huge amount of software development costs which are rarely justified given inability to actually use the device. Kipman has become an expert in working on projects that has initial oh-ah factor and media appeal but he doesn't think beyond that at all. Kinect's main use case was never gaming. It's too inaccurate for pleasant use after initial wonder fads off. Hololens's main appeal is never creating this virtual world but rather becoming a sensor that can be leveraged in all kind of applications. But that would be the least sexiest thing for Kipman to do and ofcourse that may not generate as many media articles.
foobarbecue 8y ago
Wake me up when you can use these in outdoor and dark lighting conditions. At that point they will be useful for several of my scientific needs.
JVIDEL 8y ago
So same as gglass now, how is that one doing in the enterprise market btw?

285 points by T-A 8y ago

That’s pretty exciting, especially the doubled field of view

Also selling it to the military to "increase lethality". https://www.theverge.com/2019/2/22/18236116/microsoft-hololens-army-contract-workers-letter https://www.theverge.com/2019/2/22/18236116/microsoft-holole...

ABeeSea 8y ago
Well I assume the military generally doesn’t want decreased lethality when purchasing equipment.
- setquk 8y ago
  Actually they do. Only because collateral is a PR nightmare.
  - derp_dee_derp 8y ago
    No they don't. Collateral damage is a function of inefficient lethality (killing people who are not your target is wasteful and inefficient). Increasing lethality decreases collateral.
    - setquk 8y ago
      No that's not quite right. Lethality is the effectiveness of desired outcome. Was the target eliminated? Collateral is the side effects of the desired outcome. Was anyone or anything else eliminated? They are two different metrics. Consider two contrasting analogies which break your understanding: Ballistic missile: high lethality, high collateral probability. Polonium tea: high lethality, low collateral probability. About 20 years ago, I was working in the defence sector, on guidance systems, as an engineer. Sat through many a powerpoint about how collateral could be marketed away. Turns out they won that battle eventually... Incidentally I bailed out when I saw what a fucking mess of a world I was creating.
      - derp_dee_derp 8y ago
        This is bad logic. By your metrics, the most lethal thing in the world is a rusty spoon stabbed into some ones eye; either it kills the target instantly or kills them later due to infection. Very close to 100% death rate. Very lethal. No collateral damages, either. And yet no professional militaries are armed with rusty spoons
        Dylan16807 8y ago
        I don't know what logic you're seeing but that's not an accurate description. A spoon is low lethality, low collateral.
        dragonsngoblins 8y ago
        That's because the lethality of a rusty spoon is actually very low... The lethality of having been stabbed in the eye might be high, but not of spoons themselves since it is hard to successfully wound the eye of an enemy soldier with a rusty spoon at a distance when said enemy soldier is armed with ranged weaponry, and it is still difficult to do even in melee range relative to how easy it is to wound with bayonets and combat knives.
        setquk 8y ago
        They had swords that did that. Then they found out that guns had higher lethality with lower risk to the attacker (another primary metric). Collateral wasn't much of a problem back then. No one gave a shit.
        imtringued 8y ago
        What is a bullet if not a very carefully shaped rusty spoon stabbed into someone via a complicated launching mechanism?
  - 420codebro 8y ago
    I know I'm splitting hairs, but they want lethal and accurate weapons. When they pull the trigger, they want the person they intend to "neutralize" to only be hit. They do not want to decrease their lethality. The threat of usage of a weapon (lethality) is what keeps a lot of engagements very small, or not even happening at all.
writepub 8y ago
The military also buys iPhones, and Androids. What phone do you use? That very phone was probably used to order a drone strike! Did you use GPS technology this past week? That was the tech guiding the drone to it's target. And this thing called the internet was invented to network the military
- roywiggins 8y ago
  GPS is a military technology opened to civilians... going the other way feels different, for whatever reason.
  - nradov 8y ago
    No, GPS was actually designed from the start as a dual military and civilian system. The only part that wasn't originally available to civilians was the high precision signal.
    - sdinsn 8y ago
      > GPS was actually designed from the start as a dual military and civilian system No, what prompted GPS to be opened to civilian uses was KAL007 being shot down by the Soviet Union. [1] 269 killed, including a US Rep. [1] https://en.wikipedia.org/wiki/Korean_Air_Lines_Flight_007 https://en.wikipedia.org/wiki/Korean_Air_Lines_Flight_007
      - nradov 8y ago
        No, as usual Wikipedia isn't quite accurate. Here's an actual interview with the "father of GPS". "BRAD PARKINSON: Absolutely. And there are a lot of misconceptions about that. You hear people saying, well it was a military system and eventually made a civil system. That was not true. When I testified before Congress back about 1975, I said this is a combination military civil system." https://www.sciencefriday.com/segments/how-gps-found-its-way/ https://www.sciencefriday.com/segments/how-gps-found-its-way...
- 420codebro 8y ago
int_19h 8y ago
I really don't get the opposition to this. Either we need the military, or we don't. If we do, then we want to have one that's can beat anything that could be thrown at it with as few casualties as possible. If we don't, then going after VR devices rather than the rifles is all wrong on priorities.
- s3r3nity 8y ago
  Agreed - to pretend that other countries aren't devoting the top tech talent to advance their capabilities in a similar way is beyond naive. I _want_ to agree with this "ethical opposition" approach, but I know too much game theory / history to know that I also don't want my power grid knocked out on a moment's notice because, say, Googlers opposed stronger cyber security for our military.
  - int_19h 8y ago
    I can think of specific projects that might be unethical in that sense, and cannot be justified even by "well, they're doing that". But I can't see why this one would be in that category.

yodon 8y ago

2x field of view (or possibly area of view, description could be read either way). Eye tracking. MEMS display. High grade low latency hand tracking. Improved set of default gesture recognizers. Spatial Anchors service for cross platform localizing (see same AR in same location on HoloLens, ARKit, AR Core). Some sort of geometry streaming service. Support for hardware customization (hardhats, etc). Also showed a separate stand-alone IoT Kinect + CPU/GPU hardware device.

dchichkov 8y ago

Nice. AR for the industry is an interesting field. I've worked a bit on real-time recognition and registration of CAD to point cloud, for another industrial grade AR headset, developed by another company. But unfortunately an exec from Qualcomm came in, brought in his buddies and flown the thing into the ground. Hopefully Microsoft will pick up the flag of AR for industrial use.

Barrin92 8y ago
>Nice. AR for the industry is an interesting field Yep I'm also happy about this and like the direction Microsoft seems to be going. Industrial applications have always seemed much more interesting and potentially widely beneficial than just targeting consumers and entertainment.

Klathmon 8y ago

So does anyone think this is at the point where I can pick one up and do away with multiple monitors? (Hypothetically, I see they aren't selling to consumers yet) I play a game called Elite Dangerous with my Oculus rift, and the UX of being able to have holographic windows around my person that I can look at to activate and control is so insanely powerful and so freeing, and I'm literally willing to drop multiple thousands of dollars on something right now that can give that experience with normal desktop software right now. Sadly the Rift lacks any way to see the real world when it's on (which makes using a keyboard and mouse more difficult), and the resolution still has a ways to go before I can comfortably use it like this, not to mention the software still needs work for this use case. But I can't wait until the day when I can get rid of the monitors on my desk and replace them with a headset. Being able to place normal 2d windows around my person, be able to maybe use some limited gestures to control them, and then the same keyboard and potentially mouse that in used to.

thefounder 8y ago
Both Hololens and Oculus are far from being replacements for traditional displays(i.e for programming)
- Klathmon 8y ago
  Yes, but why? What is the limiting factor here, because from my perspective it's only missing a few small things on each platform that would solve it. I'd be honestly happy with my Rift as-is if they bumped up the resolution a bit, and had some way to see the real world with the headset on (even camera passthrough is fine!). I've used virtual desktop apps in the Rift, and they aren't bad to use, and playing games like tabletop simulator is so intuitive and nice that I'm floored tht this hasn't been tried commercially yet. I assume there are some pretty significant hurtles that I'm missing, or maybe the market is still hilariously small at this point, but it just seems so close!
  - poutine 8y ago
    The resolution is way too low. Needs to be 4-16x greater for desktop replacement. Also the headsets are too hot and bulky to comfortably wear for a full workday.

Animats 8y ago

It's not a hologram, people. It's a very clever projector, but it's not a hologram at all. The example given, showing someone where to put a bolt, is about making a human do a robot's job. The computer decides what has to be done, and tells the human exactly what to do. Like the picking system in Amazon warehouses. We're getting ever closer to Marshall Brain's "Manna". Machines should think. People should work.

lwansbrough 8y ago

The piano demo was by far the most exciting for me as someone who developed for the first HoloLens. Having used the first one extensively, the improved gestures support is IMO a more significant improvement than the FOV upgrade for long term use cases. I’m interested to see how Firefox handles web browsing on HoloLens. I’ve always envisioned URLs in that context as a way of loading a new holographic experience into a shared space, as opposed to each URL providing its own separate experience (though this would be good as an option.)

roywiggins 8y ago

I played with a HoloLens 1 briefly, which was both a revelation- AR feels like it could be useful in a way VR doesn't- and very frustrating. The worst part was the gesture recognition, which was the absolute bare minimum. The Air Tap was horrible. Twice the FoV and with real hand tracking? That suddenly makes this thing viable in one step.

Groxx 8y ago
I generally feel like anything that requires your hands to be in the field of view (at anything near its current size) is an immediate non-starter, which is part of what ruined Air Tap for me. Plus it was just horribly imprecise. We do stuff with our hands largely away from the center of our vision. You might make a brief glance to calibrate yourself, but e.g. grabbing a thing doesn't require constant focus on the thing through the entire act.
- ocdtrekkie 8y ago
  This may be the least insurmountable challenge though: A pair of smart gloves will mostly solve this problem.
  - Groxx 8y ago
    yep, that'd be the most comprehensive way certainly. downward-facing cameras could probably get most of it tho.
- freeone3000 8y ago
  Air tap doesn't actually require your finger to be in field of view. Even in the Hololens 1, there is, in fact, a downward-facing camera for gestures whose field-of-camera extends all the way down to your waist.

lwansbrough 8y ago

I was disappointed to see Microsoft is still working with the concept of apps. I think we need to rethink how we start experiences in mixed reality. Pointing and tapping an app makes sense for a touch/mouse device, but not for a device that's supposed to continuously immerse the user. Here's my alternative concept: We need a search engine for holographic layer services. It would be like Google, but for MR experiences. Holographic services would use a protocol that defines a geofence to be discovered by the layer search engine's crawler over the internet (this could just be a meta tag on a classic website). The HoloLens or whatever MR device would continuously ping the search engine with its location, and the results would be ordered based on their relevance (size of geofence and proximity are good indicators). The MR device would then show the most relevant available layer in the corner of the FOV. Selecting the layer would allow enabling it either once or always, and the device would then deliver the holographic layer over the internet. The holographic layer would behave like a web service worker (in fact, it could be a web service worker) and would augment a shared experience which contains other active holographic layers. For example, your Google Maps holographic layer could be providing you with a path to walk to the nearest Starbucks, and once you're outside Starbucks, the Starbucks layer is also activated, which allows you to place an order. This concept of activated layers, I think, is a great way to avoid a future where we're being bombarded with augmented signage and unwanted experiences. In fact, you could go further and enable blocking notifications about specific/certain types of available services. (ie. don't notify me about bars or fast food restaurants.)

janoc 8y ago
That would make zero sense for Hololens - Hololens is an enterprise device that will in practice run one or two applications (e.g. the company training or maintenance tool). Also most industrial networks are behind heavy firewalls, so anything that actually depends on internet access is a problem. Hololens also doesn't really work outdoors (display contrast and tracking are the largest limitations), so there is no point of designing a system for supporting that when the hardware is not usable in such scenario.
- lwansbrough 8y ago
  It would make zero sense for HoloLens today. The HoloLens doesn't even have a GPS chip. But Microsoft is building the future of MR in general. It's a space they want to own or at least lead, and their investment in Azure services for HoloLens today are proof of that. Obviously what I've described does not really favour industrial use (though I dispute the idea that this would not be achievable in industrial/enterprise environments.) Indeed, HoloLens isn't great outdoors, however the current technology is capable of it (despite the HL design.) The visor can be darkened to function more like sunglasses, which solves the contrast problem. I'm just looking 5-10 years out, in terms of what's going to be needed.

loosetypes 8y ago

So frustrating when videos I think of as marketing on YouTube are preceded by other ads. I get that this content is from theverge, not Microsoft, but the same could be said for YT shoving ads at you when you're already just trying to watch a movie trailer. I'm sure a lot of folks rarely watch many entire segments of ads unless it's on their terms. Does further ad-walling that content not just reduce the overall number of eyeballs? It's like the only ad metrics seen as meaningful are the hostile ones..

deleted 8y ago
[deleted]
jobigoud 8y ago
My other pet peeve is movie trailers channels that start their videos with a 2-second mini trailer of the trailer.

daenz 8y ago

What's cool about the use-case they describe is that it's the perfect bridge between what humans are good at: improvised problem solving and movement, and what computers are good at: crunching and surfacing data. Is this the sweet spot?

bob_theslob646 8y ago
>Is this the sweet spot? Yes it is! What is actually hilarious about this tech is that as the hardware gets better, the information it feeds you will ultimately be the most valuable piece. Crazy to think about when verifying validity a firehouse amount of data.

___karim 8y ago

It’s a fascinating space. I’m Waiting for a startup, the ‘Apple’ of mixed reality, to take a fresh approach and create something for the rest of us.

mtgx 8y ago

For the war

kjaer 8y ago

For finger tracking, version 1 used random forests [1], because of the performance/hardware budget trade-off: they're harder to train than a traditional deep learning algorithm, but are much more efficient to compute on the device (branching being basically free on a CPU). Version 2 uses a deep learning accelerator [2], which makes it possible to do the heavier computation of DNNs (which involve floating-point operations, which would be much more expensive on the CPU). From an engineering perspective, I just love seeing how it touches all abstraction layers of the stack, and the types of solutions that come out of thinking about the silicon and the high-level ML models at the same time. [1] https://www.microsoft.com/en-us/research/wp-content/uploads/2017/05/Towards-realistic-hands-gesture-interface.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/... [2] https://homes.cs.washington.edu/~kklebeck/lebeck-tech18.pdf https://homes.cs.washington.edu/~kklebeck/lebeck-tech18.pdf

kevingadd 8y ago
Unless the paper specifically calls it out in a spot I didn't see, it's not necessarily the case that the DNN operations are floating-point. Some networks use FP16 or FP32 (it's my understanding that this is very common during training) but actual production use of a trained network can happen using int8 or int4. You can see this if you look at what the 'Tensor' cores in modern geforce cards expose support for and what Google's latest cloud tensor cores support. NV's latest cores expose small matrices of FP16, INT8 and INT4 (I've seen some suggestions that they do FP32 as well but it's not clear whether this is accurate), while Google's expose huge matrices in different formats (TPUv1 was apparently INT8, TPUv2 appears to be a mix of FP16 and FP32). In non-DNN image processing it's quite common to use ints as well (iDCT, FFT, etc) for the potential performance gains vs. floating point.
munib_ca 8y ago
When you mention version 1 and 2, are you referring to the original hololens and the new one?
- kjaer 8y ago
  Yes. I'm referring to the original HoloLens and HoloLens 2, so to the hardware versions.

telltruth 8y ago

This is quite disappointing. Lot of good people I've known who joined this team at early stage are all gone and it shows. Kipman simply didn't seem to get this. Magic Leap should have at least taught them that it is very bad idea to have so much weight on headset. The AR is pretty much dead at this point and all these push towards creating demos for factories is pure media stunt. They even managed to lure US army and I'd like to know the story of how they did this but from my own experience trying to use Hololens in such situations would be a tax, not benefit. The battery life is miserable, its too heavy for wearing on head and induces headaches for most people. Also, using it in custom situation requires huge amount of software development costs which are rarely justified given inability to actually use the device. Kipman has become an expert in working on projects that has initial oh-ah factor and media appeal but he doesn't think beyond that at all. Kinect's main use case was never gaming. It's too inaccurate for pleasant use after initial wonder fads off. Hololens's main appeal is never creating this virtual world but rather becoming a sensor that can be leveraged in all kind of applications. But that would be the least sexiest thing for Kipman to do and ofcourse that may not generate as many media articles.

foobarbecue 8y ago

Wake me up when you can use these in outdoor and dark lighting conditions. At that point they will be useful for several of my scientific needs.

JVIDEL 8y ago

So same as gglass now, how is that one doing in the enterprise market btw?