3 ms·

This is one problem where getting the results in software is very impressive, but the problem becomes much simpler with just a modicum of extra hardware. LeapM

This is one problem where getting the results in software is very impressive, but the problem becomes much simpler with just a modicum of extra hardware. LeapMotion[1] devices accomplish this with nothing more than a pair of cameras in a matchbox, in real time. And this kind of hardware is already becoming standard on cell phones and laptops. Still, the killer obstacle for the applications I was trying to make was not precision, but latency. Amazing work on HAMR - I wish they had the latency numbers in the article too! [1]https://www.leapmotion.com/ https://www.leapmotion.com/

Tade0 7y ago
Interesting. I didn't notice any significant latency when I was using it. Or were you trying to make musical instruments?
- romwell 7y ago
  >Or were you trying to make musical instruments? Bingo :) The latency was pretty low, and good for UI, but you'd want <10ms total latency for musical instruments.
  - Tade0 7y ago
    I can relate. My biggest disappointment in this regard were Android devices - such flexibility and potential! But still, they couldn't deliver. This goes to show that consumer-grade hardware(and associated software) is not up to par with the requirements of making music.
new_realist 7y ago
Last time I tried LeapMotion it was terrible. Has it improved?
amayne 7y ago
LeapMotion, the last I looked, was two stereo cameras and three IR emitters, plus their own special software algorithms. I don't think this makes the problem simpler if the problem you're trying to solve is a low-cost, extremely robust hand detection system that can be used with any device that has a camera. Those three IR emitters aren't standard on any device or category that I'm aware of. (Apple's iPhone uses an IR pattern emitter.) A software-only solution has the advantage of potentially improving dramatically with new ML models. LeapMotions's physical hardware - revolutionary when it launched – now seem like a disadvantage given the pace of human model detection (range of the IR sensors, locked into lower resolution cameras, etc.) Solving it in software...assuming there's not a performance hit, is usually the better solution, in my opinion.
paulsutter 7y ago
This isn't research about hands, this is research about recognizing 3d reality in a 2d image, filling in occluded details based on knowledge about complex objects. Hands do make a great subject for the experiment, but are not the end goal. The results are impressive and a step forward.

Tade0 7y ago

Interesting. I didn't notice any significant latency when I was using it. Or were you trying to make musical instruments?

romwell 7y ago
>Or were you trying to make musical instruments? Bingo :) The latency was pretty low, and good for UI, but you'd want <10ms total latency for musical instruments.
- Tade0 7y ago
  I can relate. My biggest disappointment in this regard were Android devices - such flexibility and potential! But still, they couldn't deliver. This goes to show that consumer-grade hardware(and associated software) is not up to par with the requirements of making music.

new_realist 7y ago

Last time I tried LeapMotion it was terrible. Has it improved?

amayne 7y ago

LeapMotion, the last I looked, was two stereo cameras and three IR emitters, plus their own special software algorithms. I don't think this makes the problem simpler if the problem you're trying to solve is a low-cost, extremely robust hand detection system that can be used with any device that has a camera. Those three IR emitters aren't standard on any device or category that I'm aware of. (Apple's iPhone uses an IR pattern emitter.) A software-only solution has the advantage of potentially improving dramatically with new ML models. LeapMotions's physical hardware - revolutionary when it launched – now seem like a disadvantage given the pace of human model detection (range of the IR sensors, locked into lower resolution cameras, etc.) Solving it in software...assuming there's not a performance hit, is usually the better solution, in my opinion.

paulsutter 7y ago

This isn't research about hands, this is research about recognizing 3d reality in a 2d image, filling in occluded details based on knowledge about complex objects. Hands do make a great subject for the experiment, but are not the end goal. The results are impressive and a step forward.