6 ms·

I think the article is a bit unfair. I've been playing around with a leap for a few days and am suitably impressed. What works very well with the device is coa

I think the article is a bit unfair. I've been playing around with a leap for a few days and am suitably impressed. What works very well with the device is coarse movements, especially relative hand movements. What doesn't work so well is finer gestures (1/100th of a millimeter motions of all 10 fingers? ha). This app is a perfect example: https://airspace.leapmotion.com/apps/cyber-science-motion https://airspace.leapmotion.com/apps/cyber-science-motion You can use your hands, kept flat, to spin around / zoom a 3-D rendering of a human skull. You can also point at specific elements on the skull. Both of these coarse gestures work great, and the experience is incredible. However, the app also unfortunately has a "click" gesture to pick apart elements of the skull - you click by spreading out your thumb and then folding it back in. Works terribly, as this fine gesture is detected maybe 50% of the time. It should've simply been left out. I showed this app to my dad, who's a doctor, and he was blown away. He was visibly excited about the potential for a device he can use to spin around CT and MRI scans in the operating room without having to touch a mouse/joystick - currently he has a person doing this for him to keep things sterile, and this can sometimes be frustrating. The leap, at least in its current incarnation, reminds me a lot of Google glass. Both Google/LEAP and their proponents say the devices are going to change the world. Maybe, maybe not. Neither device works perfectly like what you see in the heavily edited demo videos. But both can be invaluable in certain specialized fields, today, as long as folks are realistic about what can be done with them.

Keyframe 13y ago
What's the lag like?
- nairteashop 13y ago
  Not noticeable at all (I'm testing on a 2 yr old Macbook Pro).
  - Keyframe 13y ago
    I think I have a use for this if their SDK is good. I'll check it out. Thanks!
  - oscilloscope 13y ago
    I remember the latency being around 4-15ms depending on the number of hands in the frame. That's very low, but if you compared with 1-2ms latency it would be noticeable. Applications can add additional latency on top of that.
    - Keyframe 13y ago
      60Hz+ seems good for this type of tech, especially in its infancy. Obviously it would be great to have 1000Hz input like that, or even a touch screen at 1-4ms on readily available devices.
abrichr 13y ago
I showed this app to my dad, who's a doctor, and he was blown away. He was visibly excited about the potential for a device he can use to spin around CT and MRI scans in the operating room without having to touch a mouse/joystick - currently he has a person doing this for him to keep things sterile, and this can sometimes be frustrating. This is exactly the value proposition of my startup, TouchFree Labs. We're developing software that uses the Leap Motion Controller to allow surgeons to manipulate medical images inside of the operating room. You can see a demonstration of an early prototype here: http://www.youtube.com/watch?v=WaO-cimDOEQ http://www.youtube.com/watch?v=WaO-cimDOEQ. Demo starts at about 35s. (Apologies in advance for the low production value.) Right now our bottleneck is medical expertise, and we're looking for surgeons who would be interested in collaborating with us. We're developing workflows that are tailored for different types of procedures, which requires very specialized knowledge. The application also learns the nuances of individual users' movements to improve gesture recognition, which requires lots of data. I don't know how far away you are from Toronto, but if you could pass the message along to your dad, I'd be very grateful--if only to get some basic feedback. But if he's interested, he could be among the first surgeons in the world to use the Leap Motion inside of an operating room.
- nairteashop 13y ago
  Nice video, and this is a great target market. My dad's in India so collaboration could be a challenge :) PM me if you think he can still help though - I'm here on vacation for a few more weeks. BTW, not sure if you saw the following video - it was embedded in the last newsletter from Leap, and shows a vet controlling OsiriX (an open source viewer for medical/DICOM images) using Leap: https://www.facebook.com/photo.php?v=10152121411384392 https://www.facebook.com/photo.php?v=10152121411384392 Looks like a little hokey compared to cyber science or your app, but the approach of simply plugging into and enhancing an existing viewer seemed quite pragmatic.
- jamesbritt 13y ago
  Is there anything you can say about how you arranged a license for this use? The default Leap EULA forbids use with medical equipment, as well as use on a shared work-station.
emehrkay 13y ago
At this very moment I am using Apple's external trackpad. If this thing had Leap tech built in, it, to me, would be the next logical evolution of computing interfacing. I imagine the trackpad being a bit wider and the user could just raise his hands when needed, touch in just about any other situation. *I haven't used Leap yet
Qworg 13y ago
The Leap won't be a "real input device" until you can get your hands out of the flat orientation. At this point, it is a toy. I think the article is good coverage of the state of the device.
- Amadou 13y ago
  Imagine using your hand as a mouse. Just move it around on the desk next to your keyboard, tap the surface of the desk with a finger to click, you've automatically got five buttons - one for each finger, etc. I guess that would be kind of like a "virtual" trackpad plus if you need to go 3d, lift your hand off the desk.
rywang 13y ago
A big problem with the LEAP is that there isn't an effective way to click / select something. Pushing forward with your index finger isn't very accurate when the finger tip is also controlling the position. Hence, you always seem to miss where you intend to click. Good selection is a pretty important piece of almost any useful application. Disclaimer: I work at 3Gear Systems (http://threegear.com http://threegear.com), developing technology that possibly competes with the LEAP. We solve clicking by tracking the entire hand -- not just the straight finger.
- nairteashop 13y ago
  This is very interesting, doubly so because at my last gig (retailnext.net) we were looking into shopper tracking via ToF technology. Fine gestures like clicking do indeed work terribly on the Leap. Their own app (Touchless) uses a difficult "poke" gesture for clicking vs. some other coarse and easily-detectable gesture. The one thing that Leap has going for it is though is that it is small and positioned under the wrist, which means that installation is super-easy, and at some point it will be built into some laptops and keyboards. This means it will likely do better in the consumer space. (assuming they can fix all the bugs, that is) However, if your technology can detect more gestures robustly, it will do WAY better in professional environments where ease of installation is not such a big deal (e.g the surgery room, animation studios, etc). I'm sure you already know this, but just typing out my thoughts :)
  - rywang 13y ago
    A couple medical device companies and a hospital are evaluating our system right now. :-) We're actively working on supporting smaller / shorter-range sensors as well. You probably already know that in addition to the Kinect, a lot more depth cameras are on the market now: PrimeSense's Capri, SoftKinetic, PMD, Inuitive Tec, etc. All of these companies have introduced gum-stick sized sensors that can be embedded in a laptop or monitor.
- kodr 13y ago
  would it be too annoying to add another parameter for a click, like a sound of voice signal? the other alternative I'd see would be to have a virtual representation, and to be able to "click" on it, but it isn't really for a near future...
ibudiallo 13y ago
I think, from the example cited, what this author need is a touch screen lap top.
- qntmfred 13y ago
  he has one
kunikos 13y ago
Kinect for Windows already has medical applications completed for surgeons to rotate 3D imaging, as you suggest: http://research.microsoft.com/en-us/news/features/touchlesssurgery-060712.aspx http://research.microsoft.com/en-us/news/features/touchlesss... It should improve drastically when the new Kinect from Xbox One gets released for Windows.
- nobodysfool 13y ago
  Too much technology for a simple problem. Add a foot pedal controller (think of the ddr pad) and the problem is solved.

Keyframe 13y ago

What's the lag like?

nairteashop 13y ago
Not noticeable at all (I'm testing on a 2 yr old Macbook Pro).
- Keyframe 13y ago
  I think I have a use for this if their SDK is good. I'll check it out. Thanks!
- oscilloscope 13y ago
  I remember the latency being around 4-15ms depending on the number of hands in the frame. That's very low, but if you compared with 1-2ms latency it would be noticeable. Applications can add additional latency on top of that.
  - Keyframe 13y ago
    60Hz+ seems good for this type of tech, especially in its infancy. Obviously it would be great to have 1000Hz input like that, or even a touch screen at 1-4ms on readily available devices.

abrichr 13y ago

I showed this app to my dad, who's a doctor, and he was blown away. He was visibly excited about the potential for a device he can use to spin around CT and MRI scans in the operating room without having to touch a mouse/joystick - currently he has a person doing this for him to keep things sterile, and this can sometimes be frustrating. This is exactly the value proposition of my startup, TouchFree Labs. We're developing software that uses the Leap Motion Controller to allow surgeons to manipulate medical images inside of the operating room. You can see a demonstration of an early prototype here: http://www.youtube.com/watch?v=WaO-cimDOEQ http://www.youtube.com/watch?v=WaO-cimDOEQ. Demo starts at about 35s. (Apologies in advance for the low production value.) Right now our bottleneck is medical expertise, and we're looking for surgeons who would be interested in collaborating with us. We're developing workflows that are tailored for different types of procedures, which requires very specialized knowledge. The application also learns the nuances of individual users' movements to improve gesture recognition, which requires lots of data. I don't know how far away you are from Toronto, but if you could pass the message along to your dad, I'd be very grateful--if only to get some basic feedback. But if he's interested, he could be among the first surgeons in the world to use the Leap Motion inside of an operating room.

nairteashop 13y ago
Nice video, and this is a great target market. My dad's in India so collaboration could be a challenge :) PM me if you think he can still help though - I'm here on vacation for a few more weeks. BTW, not sure if you saw the following video - it was embedded in the last newsletter from Leap, and shows a vet controlling OsiriX (an open source viewer for medical/DICOM images) using Leap: https://www.facebook.com/photo.php?v=10152121411384392 https://www.facebook.com/photo.php?v=10152121411384392 Looks like a little hokey compared to cyber science or your app, but the approach of simply plugging into and enhancing an existing viewer seemed quite pragmatic.
jamesbritt 13y ago
Is there anything you can say about how you arranged a license for this use? The default Leap EULA forbids use with medical equipment, as well as use on a shared work-station.

emehrkay 13y ago

At this very moment I am using Apple's external trackpad. If this thing had Leap tech built in, it, to me, would be the next logical evolution of computing interfacing. I imagine the trackpad being a bit wider and the user could just raise his hands when needed, touch in just about any other situation. *I haven't used Leap yet

Qworg 13y ago

The Leap won't be a "real input device" until you can get your hands out of the flat orientation. At this point, it is a toy. I think the article is good coverage of the state of the device.

Amadou 13y ago
Imagine using your hand as a mouse. Just move it around on the desk next to your keyboard, tap the surface of the desk with a finger to click, you've automatically got five buttons - one for each finger, etc. I guess that would be kind of like a "virtual" trackpad plus if you need to go 3d, lift your hand off the desk.

rywang 13y ago

A big problem with the LEAP is that there isn't an effective way to click / select something. Pushing forward with your index finger isn't very accurate when the finger tip is also controlling the position. Hence, you always seem to miss where you intend to click. Good selection is a pretty important piece of almost any useful application. Disclaimer: I work at 3Gear Systems (http://threegear.com http://threegear.com), developing technology that possibly competes with the LEAP. We solve clicking by tracking the entire hand -- not just the straight finger.

nairteashop 13y ago
This is very interesting, doubly so because at my last gig (retailnext.net) we were looking into shopper tracking via ToF technology. Fine gestures like clicking do indeed work terribly on the Leap. Their own app (Touchless) uses a difficult "poke" gesture for clicking vs. some other coarse and easily-detectable gesture. The one thing that Leap has going for it is though is that it is small and positioned under the wrist, which means that installation is super-easy, and at some point it will be built into some laptops and keyboards. This means it will likely do better in the consumer space. (assuming they can fix all the bugs, that is) However, if your technology can detect more gestures robustly, it will do WAY better in professional environments where ease of installation is not such a big deal (e.g the surgery room, animation studios, etc). I'm sure you already know this, but just typing out my thoughts :)
- rywang 13y ago
  A couple medical device companies and a hospital are evaluating our system right now. :-) We're actively working on supporting smaller / shorter-range sensors as well. You probably already know that in addition to the Kinect, a lot more depth cameras are on the market now: PrimeSense's Capri, SoftKinetic, PMD, Inuitive Tec, etc. All of these companies have introduced gum-stick sized sensors that can be embedded in a laptop or monitor.
kodr 13y ago
would it be too annoying to add another parameter for a click, like a sound of voice signal? the other alternative I'd see would be to have a virtual representation, and to be able to "click" on it, but it isn't really for a near future...

ibudiallo 13y ago

I think, from the example cited, what this author need is a touch screen lap top.

qntmfred 13y ago
he has one

kunikos 13y ago

Kinect for Windows already has medical applications completed for surgeons to rotate 3D imaging, as you suggest: http://research.microsoft.com/en-us/news/features/touchlesssurgery-060712.aspx http://research.microsoft.com/en-us/news/features/touchlesss... It should improve drastically when the new Kinect from Xbox One gets released for Windows.

nobodysfool 13y ago
Too much technology for a simple problem. Add a foot pedal controller (think of the ddr pad) and the problem is solved.