7 ms·

Show HN: Touchable augmented reality on your phone, 3 years in the making

15 points by lazyjeff 7y ago

villgax 7y ago
Try checking out https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html https://ai.googleblog.com/2019/08/on-device-real-time-hand-t... this might remove the need for any hardware apart from the phone & its vibration motor.
- deleted 7y ago
  [deleted]
- lazyjeff 7y ago
  Yup we've looked into that. It's not accurate enough to feel good to use, and often you go off screen for a bit which breaks the interaction. The infrared sensor actually has a field of view wider than the camera and helps maintain the connection in that brief offscreen moment.
Phillips126 7y ago
Looks quite interesting. I remember when the Leap Motion was first being released but I haven't seen a great deal in the wild that use it. I signed up to get an early dev kit of their technology when I was working at an old job. We were a non-profit agency that helped educate individuals with disabilities - often having difficulty with their motor skills (using a typical mouse/keyboard was impossible for most of them). The idea was we'd use Unity3D or a similar game engine to create minigames that used the leap motion as the input device. The games would educate the individuals on many things from normal daily hygiene to cooking, etc. In the end, our tiny team (of 3-4 people) bit off more than we could chew and nothing substantial was ever materialized. The leader of the team moved on to another company and the project dissolved. It was definitely one of the more interesting projects I worked on at that company.
akrymski 7y ago
Cool. But you shouldn't need Leap for this these days, you can do hand tracking with ML: https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html https://ai.googleblog.com/2019/08/on-device-real-time-hand-t...
- lazyjeff 7y ago
  We have directly tested the MediaPipe one and it's less accurate than the Leap. Like it's completely unusable in practice, especially when the palm is not facing the camera. Also there's an HCI problem, which is that people often go offscreen when interacting with objects (your brain assumes things can happen outside the field of view), but even a moment outside the field of view breaks the interaction.

15 points by lazyjeff 7y ago

Try checking out https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html https://ai.googleblog.com/2019/08/on-device-real-time-hand-t... this might remove the need for any hardware apart from the phone & its vibration motor.

deleted 7y ago
[deleted]
lazyjeff 7y ago
Yup we've looked into that. It's not accurate enough to feel good to use, and often you go off screen for a bit which breaks the interaction. The infrared sensor actually has a field of view wider than the camera and helps maintain the connection in that brief offscreen moment.

Phillips126 7y ago

Looks quite interesting. I remember when the Leap Motion was first being released but I haven't seen a great deal in the wild that use it. I signed up to get an early dev kit of their technology when I was working at an old job. We were a non-profit agency that helped educate individuals with disabilities - often having difficulty with their motor skills (using a typical mouse/keyboard was impossible for most of them). The idea was we'd use Unity3D or a similar game engine to create minigames that used the leap motion as the input device. The games would educate the individuals on many things from normal daily hygiene to cooking, etc. In the end, our tiny team (of 3-4 people) bit off more than we could chew and nothing substantial was ever materialized. The leader of the team moved on to another company and the project dissolved. It was definitely one of the more interesting projects I worked on at that company.

akrymski 7y ago

Cool. But you shouldn't need Leap for this these days, you can do hand tracking with ML: https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html https://ai.googleblog.com/2019/08/on-device-real-time-hand-t...

lazyjeff 7y ago
We have directly tested the MediaPipe one and it's less accurate than the Leap. Like it's completely unusable in practice, especially when the palm is not facing the camera. Also there's an HCI problem, which is that people often go offscreen when interacting with objects (your brain assumes things can happen outside the field of view), but even a moment outside the field of view breaks the interaction.