7 ms·

I have actually developed on the Leap Motion device and here are my few critiques: 1. There are way too many gestures and motions available. The SDK needs to p

I have actually developed on the Leap Motion device and here are my few critiques: 1. There are way too many gestures and motions available. The SDK needs to provide a solid framework for fewer gestures/motions and the company needs to make sure that consumers readily understand them. Unless there is a whole lot of standardization of the platform it will become hard for developers/other companies to produce a consistent and coherent experience. 2. I do not like the strategy of lets-throw-it-in-the-wild-and-see-what-other-people-come-up-with; the tech has loads of cool factor that you will get people to make interesting stuff with it. However, will that translate into useable and effective computing software? From my point of view, I have not really seen a single application on Leap which does something 'better' than a touch based or mouse-based interface can do. Better meaning more intuitively and/or more productively 3. I think one of the biggest challenges for any gesture controlled device is the shoulder/forearm/wrist/palm pain that is inherently going to be the result of prolonged gesture-driven interface use. Maybe applications have to use gestures sparingly? On a broader note, I still think humans have an inherent desire to touch/feel interface elements. Being able to directly touch a screen and manipulate it means I can easily manipulate visual behavior through an actual/physical connection. Interfaces which disconnect this physical medium seem more alien to me at least. Maybe I am less evolved as a neanderthal but I don't think we are ready for a gesture driven interface.

mtrimpe 13y ago
I'm mostly just surprised that it seems like they didn't include a trainable gesture recognition app. It should be pretty doable to hook it up to a neural network and train it to recognize some of your own gestures. I was already looking forward to training up a drumroll gesture for 'deploy to production.'
- msutherl 13y ago
  Prototype it in a couple of days with FTM[1] for Max[2]. Video[3]. [1] http://ftm.ircam.fr/index.php/Gesture_Follower http://ftm.ircam.fr/index.php/Gesture_Follower [2] http://cycling74.com/products/max/ http://cycling74.com/products/max/ [3] https://www.youtube.com/watch?v=2dSU8MeJlz4 https://www.youtube.com/watch?v=2dSU8MeJlz4
- andrewflnr 13y ago
  To prevent accidental deploys, it waits for a second of drumrolling, then asks if you're sure you want to deploy. To confirm, hit the cymbal.
jre 13y ago
I think there are a number of specific areas where Leap Motion can be interesting. For example, there are some startups that use Kinect to let surgeon interact with a screen showing patient data during an operation. Some researchers have also proposed the use of gesture recognition in the context of a worker having to follow precise instructions for some complex task. Leap motion + Google Glass could be an interesting combo for interactive training. I'm sure there are other areas. But I agree with your comment and I fail to see the mass market use case. But I'd be happy to be proven wrong :-)
- ahsanhilal 13y ago
  I think your right, that it might be prove useful in edge cases and custom application scenarios. But as a consumer product, it's just confusing at the moment. I make mobile applications for kids and I made a prototype of one of our games for some kids to work with, but even after a tutorial it was hard for kids to pick it up. I think there is something very human about being able to physically interact with things. And it is extremely hard to break that barrier.
- irickt 13y ago
  Leap wants an unspecified cut if you make one of these: "Specialized Application" means an Application which is: (i) sold, licensed, leased, or otherwise disposed of for a list price of more than US$500 or local equivalent, or more than US$240 per year or local equivalent if on a subscription, lease or similar basis; (ii) sold, licensed, leased or otherwise disposed of as part of, or for use with, another application, system, machine or device (other than a personal computer), having a list price of more than US$500 or local equivalent, or more than US$240 per year or local equivalent if on a subscription, lease or similar basis; (iii) designed for use, or that is primarily used, with or for control, whether direct or indirect, of industrial, commercial, military or medical equipment.
  - voltagex_ 13y ago
    Suddenly, the device seems a lot less interesting.
  - 13y ago
oscilloscope 13y ago
Your third complaint can be resolved by placing the device lower than your elbow. Most people use it on the desk, which is the problem. It's not the ideal height for the Leap. When it's higher than your elbow, your shoulder must be engaged and power all your movements. There are also many apps that map directly from x/y/z space. These can be hard to navigate. All your joints move in arcs, not straight lines. You get to use the Leap however you want, so figure out what's comfortable for yourself. Leap is quite sensitive. You could look for tiny flicks of the finger, for instance to create a virtual DataHand. These kinds of interfaces should be much gentler on the body than apps which require large sweeping motions, punches, etc. https://en.wikipedia.org/wiki/Datahand https://en.wikipedia.org/wiki/Datahand
- ahsanhilal 13y ago
  I don't think a lot of consumers will be ok with trying to find the optimal place to fit a Leap on their computer desks. It is a forced behavior not covering any specific need. Leap has a long way to go in defining a use case for the technology; something which it does better than existing interfaces. Otherwise it will be a really cool niche product used by tech enthusiasts, not something a heavily VC-backed form is targetting. Also, there is a reason DataHands never really took off and it is the same reason Leap might not. Changing human behavior is ridiculously hard, and pouring money in a dev ecosystem (how Leap is doing it) is not the way to do it.
  - TheLegace 13y ago
    I personally think it should be additional input to whatever, for your desktop or laptop. I'm planning on creating my own laptop startup and eventually try to integrate Leap(or something similar). In many cases it may not apply to general usage, but there are many use cases where it would be preferred. 1) Cases where user interacts with a GUI where user is visually engaged, imagine I am on Youtube, personally because stupid instability of HTML5 and Flash on Linux, I hate using it, especially when I am full screening. That's something that can be replaced with a gesture. So try doing a pinch gesture(basically zoom out) and the browser can interrupt that as put video in full screen. 2) Rather than using your whole arm what about just one finger. Lets say you want to scroll down and both hands are on the keyboard, rather than moving one hand back to the mouse you can just flick one finger. This gesture could scroll pages, it could flip through tabs and you wouldn't even have to take your hands of the keyboard. 3) Now there are a lot of edge cases I would say where using a keyboard/trackpad is uncomfortable. When I am lying down using a trackpad is very awkward and strains my wrist. This is certainly the case where I would want to flick some stuff around, maybe even talk to the computer. 4) My parents connect their laptop via HDMI cable to the TV and that's all they use the laptop for, basically to watch youtube and Netflix(if Leap motion had the range), it's something I would want to control almost as an imaginary but intuitive remote. And this is kind of just the start. I think with the increasing number of input devices available interfaces are going to have to be clever to hide that complexity from the user and make things intuitive.
kwindla 13y ago
We're inventing this stuff as we go, right now, which makes gestural and spatial interfaces an exciting thing to work on. Your broader point is definitely well taken. Touch is important. But touch, combined with free-space gesture, plus appropriate use of tools (like the keyboard), and voice, and head tracking, and other kinds of contextual sensing from our computers -- now that starts to look like the future. I work at Oblong Industries. We're the folks who did the interfaces in Minority Report. We've been building gestural and spatial stuff for a long time. Regarding (1) - standardization is really important. I'm old enough to remember the early days of mouse-driven interfaces. It took a long time for the standard window manipulation semantics, scrollbars, buttons, and drop-down menus to develop. The original Macintosh team did a lot of quite elegant heavy lifting in that regard. It's very much worth reading the stories on Andy Hertzfeld's site and in the related book. http://www.folklore.org/index.py http://www.folklore.org/index.py ; http://www.amazon.com/Revolution-The-Valley-Insanely-Great/dp/1449316247 http://www.amazon.com/Revolution-The-Valley-Insanely-Great/d... If you'd like to experiment with a toolkit that provides a standard framework for building gestural, spatial, multi-screen, multi-device applications, please feel free check out our Greenhouse SDK. Greenhouse supports the Leap Motion controller, the Kinect, and a bunch of other forms of input. http://greenhouse.oblong.com/ http://greenhouse.oblong.com/ Regarding (3) - we've got a lot of experience now with ergonomics of spatial interfaces. (We've been selling glove-based gestural systems to early adopter bigco customers since 2005.) It turns out that accuracy of the underlying sensing hardware is incredibly important, and then on top of that design of the gesture language is really important, too. Basically, if you have good enough sensing that small motions are precisely tracked, and then if you use that accuracy to track hand movements and poses that are "natural" (glossing over what "natural" means, for the moment), people can comfortably and happily use gestural interfaces all day, every day. Think about it this way: most people "talk with their hands" all the time. (Some people wave their hands around more than others. But spend some time paying attention to people moving their hands while they talk; it's really interesting to watch.) Our high-end, glove-based systems track finger positions to 0.01 mm at 100hz. The tracking volume is big, so your motions aren't constrained. You can walk around and you can go right up to a screen or stand back from it. We recognize hand poses ("one finger point," "two finger point," etc.) "Talking" to the computer with your hands feels a lot like what you do when you use your hands while making a point in a discussion with another person. You don't get tired and you don't feel any strain or overuse pain. The consumer-priced sensors like the Leap Motion and the Kinect don't yet provide this combination of tracking precision, tracking volume, and robustness to occlusion. But they're getting closer! And when a $50 sensor (or a few $0.50 sensors) give you the same accuracy and precision we have today with Oblong's optically tracked gloves, our collective expectations about interfaces and user experiences are going to radically change. (Or, more accurately, as happened with the invention of the GUI we use today, our expectations will change slowly over fifteen years, starting roughly now, with some moments of punctuated equilibrium analogous to the release of the Mac in 1984 and Windows 95 in, um, 1995.)
heuermh 13y ago
I have also developed for the Leap Motion device [1] and agree with much of your critique. The only arrangement I have had real success with is far away from the screen, i.e. mounted in a guitar. My kids have been mildly amused by some of the things I've written and by the apps available in the beta, but e.g. with Cut the Rope, a lot of fun swiping on touchscreen iOS devices, they couldn't get it to work waving about in empty space with the Leap Motion. "Dad, this is too hard." [1] https://github.com/heuermh/leap-motion-processing https://github.com/heuermh/leap-motion-processing

mtrimpe 13y ago

I'm mostly just surprised that it seems like they didn't include a trainable gesture recognition app. It should be pretty doable to hook it up to a neural network and train it to recognize some of your own gestures. I was already looking forward to training up a drumroll gesture for 'deploy to production.'

msutherl 13y ago
Prototype it in a couple of days with FTM[1] for Max[2]. Video[3]. [1] http://ftm.ircam.fr/index.php/Gesture_Follower http://ftm.ircam.fr/index.php/Gesture_Follower [2] http://cycling74.com/products/max/ http://cycling74.com/products/max/ [3] https://www.youtube.com/watch?v=2dSU8MeJlz4 https://www.youtube.com/watch?v=2dSU8MeJlz4
andrewflnr 13y ago
To prevent accidental deploys, it waits for a second of drumrolling, then asks if you're sure you want to deploy. To confirm, hit the cymbal.

jre 13y ago

I think there are a number of specific areas where Leap Motion can be interesting. For example, there are some startups that use Kinect to let surgeon interact with a screen showing patient data during an operation. Some researchers have also proposed the use of gesture recognition in the context of a worker having to follow precise instructions for some complex task. Leap motion + Google Glass could be an interesting combo for interactive training. I'm sure there are other areas. But I agree with your comment and I fail to see the mass market use case. But I'd be happy to be proven wrong :-)

ahsanhilal 13y ago
I think your right, that it might be prove useful in edge cases and custom application scenarios. But as a consumer product, it's just confusing at the moment. I make mobile applications for kids and I made a prototype of one of our games for some kids to work with, but even after a tutorial it was hard for kids to pick it up. I think there is something very human about being able to physically interact with things. And it is extremely hard to break that barrier.
irickt 13y ago
Leap wants an unspecified cut if you make one of these: "Specialized Application" means an Application which is: (i) sold, licensed, leased, or otherwise disposed of for a list price of more than US$500 or local equivalent, or more than US$240 per year or local equivalent if on a subscription, lease or similar basis; (ii) sold, licensed, leased or otherwise disposed of as part of, or for use with, another application, system, machine or device (other than a personal computer), having a list price of more than US$500 or local equivalent, or more than US$240 per year or local equivalent if on a subscription, lease or similar basis; (iii) designed for use, or that is primarily used, with or for control, whether direct or indirect, of industrial, commercial, military or medical equipment.
- voltagex_ 13y ago
  Suddenly, the device seems a lot less interesting.
- 13y ago

oscilloscope 13y ago

Your third complaint can be resolved by placing the device lower than your elbow. Most people use it on the desk, which is the problem. It's not the ideal height for the Leap. When it's higher than your elbow, your shoulder must be engaged and power all your movements. There are also many apps that map directly from x/y/z space. These can be hard to navigate. All your joints move in arcs, not straight lines. You get to use the Leap however you want, so figure out what's comfortable for yourself. Leap is quite sensitive. You could look for tiny flicks of the finger, for instance to create a virtual DataHand. These kinds of interfaces should be much gentler on the body than apps which require large sweeping motions, punches, etc. https://en.wikipedia.org/wiki/Datahand https://en.wikipedia.org/wiki/Datahand

ahsanhilal 13y ago
I don't think a lot of consumers will be ok with trying to find the optimal place to fit a Leap on their computer desks. It is a forced behavior not covering any specific need. Leap has a long way to go in defining a use case for the technology; something which it does better than existing interfaces. Otherwise it will be a really cool niche product used by tech enthusiasts, not something a heavily VC-backed form is targetting. Also, there is a reason DataHands never really took off and it is the same reason Leap might not. Changing human behavior is ridiculously hard, and pouring money in a dev ecosystem (how Leap is doing it) is not the way to do it.
- TheLegace 13y ago
  I personally think it should be additional input to whatever, for your desktop or laptop. I'm planning on creating my own laptop startup and eventually try to integrate Leap(or something similar). In many cases it may not apply to general usage, but there are many use cases where it would be preferred. 1) Cases where user interacts with a GUI where user is visually engaged, imagine I am on Youtube, personally because stupid instability of HTML5 and Flash on Linux, I hate using it, especially when I am full screening. That's something that can be replaced with a gesture. So try doing a pinch gesture(basically zoom out) and the browser can interrupt that as put video in full screen. 2) Rather than using your whole arm what about just one finger. Lets say you want to scroll down and both hands are on the keyboard, rather than moving one hand back to the mouse you can just flick one finger. This gesture could scroll pages, it could flip through tabs and you wouldn't even have to take your hands of the keyboard. 3) Now there are a lot of edge cases I would say where using a keyboard/trackpad is uncomfortable. When I am lying down using a trackpad is very awkward and strains my wrist. This is certainly the case where I would want to flick some stuff around, maybe even talk to the computer. 4) My parents connect their laptop via HDMI cable to the TV and that's all they use the laptop for, basically to watch youtube and Netflix(if Leap motion had the range), it's something I would want to control almost as an imaginary but intuitive remote. And this is kind of just the start. I think with the increasing number of input devices available interfaces are going to have to be clever to hide that complexity from the user and make things intuitive.

kwindla 13y ago

We're inventing this stuff as we go, right now, which makes gestural and spatial interfaces an exciting thing to work on. Your broader point is definitely well taken. Touch is important. But touch, combined with free-space gesture, plus appropriate use of tools (like the keyboard), and voice, and head tracking, and other kinds of contextual sensing from our computers -- now that starts to look like the future. I work at Oblong Industries. We're the folks who did the interfaces in Minority Report. We've been building gestural and spatial stuff for a long time. Regarding (1) - standardization is really important. I'm old enough to remember the early days of mouse-driven interfaces. It took a long time for the standard window manipulation semantics, scrollbars, buttons, and drop-down menus to develop. The original Macintosh team did a lot of quite elegant heavy lifting in that regard. It's very much worth reading the stories on Andy Hertzfeld's site and in the related book. http://www.folklore.org/index.py http://www.folklore.org/index.py ; http://www.amazon.com/Revolution-The-Valley-Insanely-Great/dp/1449316247 http://www.amazon.com/Revolution-The-Valley-Insanely-Great/d... If you'd like to experiment with a toolkit that provides a standard framework for building gestural, spatial, multi-screen, multi-device applications, please feel free check out our Greenhouse SDK. Greenhouse supports the Leap Motion controller, the Kinect, and a bunch of other forms of input. http://greenhouse.oblong.com/ http://greenhouse.oblong.com/ Regarding (3) - we've got a lot of experience now with ergonomics of spatial interfaces. (We've been selling glove-based gestural systems to early adopter bigco customers since 2005.) It turns out that accuracy of the underlying sensing hardware is incredibly important, and then on top of that design of the gesture language is really important, too. Basically, if you have good enough sensing that small motions are precisely tracked, and then if you use that accuracy to track hand movements and poses that are "natural" (glossing over what "natural" means, for the moment), people can comfortably and happily use gestural interfaces all day, every day. Think about it this way: most people "talk with their hands" all the time. (Some people wave their hands around more than others. But spend some time paying attention to people moving their hands while they talk; it's really interesting to watch.) Our high-end, glove-based systems track finger positions to 0.01 mm at 100hz. The tracking volume is big, so your motions aren't constrained. You can walk around and you can go right up to a screen or stand back from it. We recognize hand poses ("one finger point," "two finger point," etc.) "Talking" to the computer with your hands feels a lot like what you do when you use your hands while making a point in a discussion with another person. You don't get tired and you don't feel any strain or overuse pain. The consumer-priced sensors like the Leap Motion and the Kinect don't yet provide this combination of tracking precision, tracking volume, and robustness to occlusion. But they're getting closer! And when a $50 sensor (or a few $0.50 sensors) give you the same accuracy and precision we have today with Oblong's optically tracked gloves, our collective expectations about interfaces and user experiences are going to radically change. (Or, more accurately, as happened with the invention of the GUI we use today, our expectations will change slowly over fifteen years, starting roughly now, with some moments of punctuated equilibrium analogous to the release of the Mac in 1984 and Windows 95 in, um, 1995.)

heuermh 13y ago

I have also developed for the Leap Motion device [1] and agree with much of your critique. The only arrangement I have had real success with is far away from the screen, i.e. mounted in a guitar. My kids have been mildly amused by some of the things I've written and by the apps available in the beta, but e.g. with Cut the Rope, a lot of fun swiping on touchscreen iOS devices, they couldn't get it to work waving about in empty space with the Leap Motion. "Dad, this is too hard." [1] https://github.com/heuermh/leap-motion-processing https://github.com/heuermh/leap-motion-processing