3 ms·

I almost made a comment in this article about using Deep Learning and Motion Tracking to have computers better understand our gestures so we can interact visual

I almost made a comment in this article about using Deep Learning and Motion Tracking to have computers better understand our gestures so we can interact visually. https://news.ycombinator.com/item?id=21115863 https://news.ycombinator.com/item?id=21115863 At some point AR and VR will finally provide us the overwhelming need. Then we’ll wonder why we didn’t think of it sooner. At the moment, we’re stuck in the “a keyboard and mouse are better” stage.

weego 7y ago
You can't just drop the idea of Deep Learning into a problem as a magical solution. What does visual interaction entail and what is stopping that from having already been trialled during one of the previous (and current) attempts to get vr to work for people?
- melling 7y ago
  If you can use Deep Learning and CNN’s to drive a car, you can probably use it to recognize different hand motions. [Update] I found a link where someone tried to do a simplified version of this: https://towardsdatascience.com/tutorial-using-deep-learning-and-cnns-to-make-a-hand-gesture-recognition-model-371770b63a51 https://towardsdatascience.com/tutorial-using-deep-learning-...
  - mkl 7y ago
    It's certainly possible to recognise hand motions (see e.g. Leap Motion), but I think hand motion recognition is pretty hard to use effectively without haptic feedback telling you what you're touching and where. Ideally actuated like [1] but for your whole hand, not just a stylus. I don't think technology is anywhere near that. The device in [1] pushes back against you, physically stopping you from moving through virtual objects. [1] https://www.3dsystems.com/haptics-devices/touch https://www.3dsystems.com/haptics-devices/touch
    - erikpukinskis 7y ago
      You don’t need haptic feedback, just good feedback. There’s plenty of bits in an audio stream alone to give the brain the control feedback it needs. You’d be surprised how few degrees of freedom we use for grasping.

weego 7y ago

You can't just drop the idea of Deep Learning into a problem as a magical solution. What does visual interaction entail and what is stopping that from having already been trialled during one of the previous (and current) attempts to get vr to work for people?

melling 7y ago
If you can use Deep Learning and CNN’s to drive a car, you can probably use it to recognize different hand motions. [Update] I found a link where someone tried to do a simplified version of this: https://towardsdatascience.com/tutorial-using-deep-learning-and-cnns-to-make-a-hand-gesture-recognition-model-371770b63a51 https://towardsdatascience.com/tutorial-using-deep-learning-...
- mkl 7y ago
  It's certainly possible to recognise hand motions (see e.g. Leap Motion), but I think hand motion recognition is pretty hard to use effectively without haptic feedback telling you what you're touching and where. Ideally actuated like [1] but for your whole hand, not just a stylus. I don't think technology is anywhere near that. The device in [1] pushes back against you, physically stopping you from moving through virtual objects. [1] https://www.3dsystems.com/haptics-devices/touch https://www.3dsystems.com/haptics-devices/touch
  - erikpukinskis 7y ago
    You don’t need haptic feedback, just good feedback. There’s plenty of bits in an audio stream alone to give the brain the control feedback it needs. You’d be surprised how few degrees of freedom we use for grasping.