8 ms·

Tiny Radar Chip Revolutionizing Gesture Recognition: Google ATAP’s Project Soli

112 points by elijahparker 10y ago

Aelinsaar 10y ago
This is really exciting. It's sort of clear that there is no one killer interface, but truly robust gesture recognition, DL driven natural language interfaces, and speech recognition start to describe a nice landscape for VR and AR. When I'm trying to create a lot of text, I'll still use a keyboard, but most of the time that's not what I want to be doing.
- melling 10y ago
  Yeah, most developers say they'll stick with a keyboard. I'm still holding out for the clever person who integrates gestures into a keyboard killer. You can use gestures chords, for example. With intellisense and other intelligent completion, the future might not need a keyboard. Here are some other gesture solutions: https://github.com/melling/ErgonomicNotes#gesture-computing https://github.com/melling/ErgonomicNotes#gesture-computing
  - Aelinsaar 10y ago
    Soli in particular seems like it could be a huge driver for this, given that the market is "Mobile devices". I'm hugely excited.
  - shostack 10y ago
    I wonder if they have to be exclusive. How cool would it be to have a keyboard where you can hold down a "gesture button" or just move your hands/fingers over a certain zone and have it pick up small gestures? You'd barely need to move your hands and could probably keep your wrists in the same position. This could then do things like more natural scrolling, cursor control, obviously more complex actions. They keyboard is great for what it does, but improving input doesn't necessarily have to mean getting rid of something that works quite well and completely reimagining it.
    - Aelinsaar 10y ago
      What has probably seen the end of the road, is the mouse.
      - Natanael_L 10y ago
        Not yet. There's few things matching its combination of speed and precision. It's more believable that computer mice will get tech like Soli integrated to become smarter.
        Aelinsaar 10y ago
        So... a mouse and a motion control transceiver in one? That could be fun.
  - andrewflnr 10y ago
    I don't think automatic completion is ever going to be a good substitute for solid text input. If your text input solution always requires it, I for one am not buying. That said, I'm hopeful for a gesture-based text input system that doesn't require any crutches.
jmount 10y ago
In my opinion this is awful for those of us who have to help people do things from time to time. Yet another UI that is impossible to walk somebody through over the phone.
- Numberwang 10y ago
  I'm working in tech support and your post just made my hair turn white.
- WhoBeI 10y ago
  Yes, well, at least now you can have your computer shut down by giving it the finger.
mrfusion 10y ago
This might be great for vr? How does it compare to leap motion?
peter303 10y ago
I though the auto industry was commoditizing this technology with minature and cheap technology. This also happened with air bag accelerometers which sell for dimes now. They were first used to detect falling laptops and retract rotating disk arms to save disks. Then you put two or three in mobile devices figure out the 3D orientation of screens.
- sheraz 10y ago
  I always wondered how the phone knew its orientation. Then I saw this video. Wow. https://www.youtube.com/watch?v=KZVgKu6v808 https://www.youtube.com/watch?v=KZVgKu6v808 This is from a series of videos called "Engineerguy"
  - mrestko 10y ago
    His videos are wonderful.
intrasight 10y ago
Cool. So when can we get our hands on (or should I say hands off) developer kits?
Animats 10y ago
Small radar ICs will be useful for other things. Radar gives you range rate directly. Getting range rate from vision is hard and takes several frame times at least. If you have position and range rate and dump it into a machine learning system, gesture recognition should be straightforward. It's not clear how useful this will be for GUIs in general. It could be useful for interacting with large screen displays, where you may be too far back for touch and a mouse is a pain. A presentation version could be useful. (Presentations from computers still look lame. The audience usually sees the presenter's desktop while they futz with the computer, then enter PowerPoint.)
- semi-extrinsic 10y ago
  > Presentations from computers still look lame. The audience usually sees the presenter's desktop while they futz with the computer, then enter PowerPoint. 1. Unplug VGA/HDMI cable 2. Do all of the futzing 3. Plug VGA/HDMI cable This is a solved technical problem. That people can't be bothered is the real problem.
  - kevhito 10y ago
    Alternative if you present regularly: 1. Enable "extended destkop", rather than "mirror desktop". 2. Select an appropriate wallpaper, or just black.
    - digi_owl 10y ago
      Best i can tell, presenters use mirror so that they can see their own slides on the screen while talking. Frankly i think a seldom talked about Android feature may be a better solution. https://developer.android.com/about/versions/android-4.2.html#SecondaryDisplays https://developer.android.com/about/versions/android-4.2.htm...
      - anc84 10y ago
        Any presentation tool worth using has a presenter mode where on your primary display you see things like the current slide, your notes, a timer, the upcoming slide, etc.
neurotech1 10y ago
They've had "small" radar modules for a while, commonly used as microwave movement detectors[0] Also, here is the manufacturer brochure[1] [0] http://dangerousprototypes.com/blog/2013/08/21/breakout-board-uses-hb100-doppler-motion-sensor/ http://dangerousprototypes.com/blog/2013/08/21/breakout-boar... [1] http://www.agilsense.com/useruploads/files/01_HB%20selection%20list%20ver300.pdf http://www.agilsense.com/useruploads/files/01_HB%20selection...
- ChuckMcM 10y ago
  The difference being Multi-Antenna Synthetic Aperture RADAR vs single antenna RADAR.
  - wyager 10y ago
    Precisely. You need synthetic aperture (or similarly complicated Fourier analysis) with multiple transceive pathways to get a good 3D model of the environment. It's very impressive that this project managed to miniaturize both the radio hardware and the computational hardware required for this. You need to perform FFTs very fast to make this work.
krapht 10y ago
I wish there was more technical detail. I would love to know the performance characteristics of the radar, and what techniques it uses (eg. is it fully coherent, waveform repetition frequency, etc).
trhway 10y ago
so they are using something like 60-66GHz which beside small antenna size advantage makes for the fastest attenuation http://www.propagation.gatech.edu/ECE6390/project/Fall2012/Team09/Team9GeoSatTech_website_FINAL/SatCom%20website/images/GaseousAttenuation.png http://www.propagation.gatech.edu/ECE6390/project/Fall2012/T... . Pretty logical.
jhallenworld 10y ago
Many alarm systems have gunn-diode microwave motion sensors in them. Replace this with high-resolution radar and networking for house-wide whole-body gesture recognition.
Animats 10y ago
If this is available with a few meters of range, it has potential for automatic door control. Star Trek quality door control has been done experimentally [1] but the standard sensors are too dumb. You want range, azimuth and range rate. This is no big deal in a radar; it just costs too much. If someone is moving fast, the door opens sooner, and faster. Moving parallel to the door should not trigger it. (In Japan, automatic sliding doors on street storefronts are common, and false triggers from passers-by are a problem.) With good sensors on both sides, you can open the door further in advance if there's someone on the other side. This allows opaque automatic doors. [1] https://www.youtube.com/watch?v=WhrLetrTSYA https://www.youtube.com/watch?v=WhrLetrTSYA
- jdhawk 10y ago
  60Ghz radio, 2-3 foot range. Soli's sensors can detect motion at a range of about two to three feet, Schwesig says, so any device you use within that range stands to benefit. http://mashable.com/2015/05/30/google-project-soli-analysis/#I.OrHXCphqqm http://mashable.com/2015/05/30/google-project-soli-analysis/...
internaut 10y ago
I want to combine one of these chips with a magic mirror. If this works, then it is one of Google's greatest projects, the number of applications are huge.
dudifordMann 10y ago
Another great tool for touch-less interfaces. I think the odd aspect of this Google I/O demonstration is the novelty usage of the technology for a smart watch. This tech seems like it is more difficult than it's worth in that form factor, but for devices that must remain sanitary, say in a kitchen, or a medical device in an OR, I like the potential.

112 points by elijahparker 10y ago

This is really exciting. It's sort of clear that there is no one killer interface, but truly robust gesture recognition, DL driven natural language interfaces, and speech recognition start to describe a nice landscape for VR and AR. When I'm trying to create a lot of text, I'll still use a keyboard, but most of the time that's not what I want to be doing.

melling 10y ago
Yeah, most developers say they'll stick with a keyboard. I'm still holding out for the clever person who integrates gestures into a keyboard killer. You can use gestures chords, for example. With intellisense and other intelligent completion, the future might not need a keyboard. Here are some other gesture solutions: https://github.com/melling/ErgonomicNotes#gesture-computing https://github.com/melling/ErgonomicNotes#gesture-computing
- Aelinsaar 10y ago
  Soli in particular seems like it could be a huge driver for this, given that the market is "Mobile devices". I'm hugely excited.
- shostack 10y ago
  I wonder if they have to be exclusive. How cool would it be to have a keyboard where you can hold down a "gesture button" or just move your hands/fingers over a certain zone and have it pick up small gestures? You'd barely need to move your hands and could probably keep your wrists in the same position. This could then do things like more natural scrolling, cursor control, obviously more complex actions. They keyboard is great for what it does, but improving input doesn't necessarily have to mean getting rid of something that works quite well and completely reimagining it.
  - Aelinsaar 10y ago
    What has probably seen the end of the road, is the mouse.
    - Natanael_L 10y ago
      Not yet. There's few things matching its combination of speed and precision. It's more believable that computer mice will get tech like Soli integrated to become smarter.
      - Aelinsaar 10y ago
        So... a mouse and a motion control transceiver in one? That could be fun.
- andrewflnr 10y ago
  I don't think automatic completion is ever going to be a good substitute for solid text input. If your text input solution always requires it, I for one am not buying. That said, I'm hopeful for a gesture-based text input system that doesn't require any crutches.

jmount 10y ago

In my opinion this is awful for those of us who have to help people do things from time to time. Yet another UI that is impossible to walk somebody through over the phone.

Numberwang 10y ago
I'm working in tech support and your post just made my hair turn white.
WhoBeI 10y ago
Yes, well, at least now you can have your computer shut down by giving it the finger.

mrfusion 10y ago

This might be great for vr? How does it compare to leap motion?

peter303 10y ago

I though the auto industry was commoditizing this technology with minature and cheap technology. This also happened with air bag accelerometers which sell for dimes now. They were first used to detect falling laptops and retract rotating disk arms to save disks. Then you put two or three in mobile devices figure out the 3D orientation of screens.

sheraz 10y ago
I always wondered how the phone knew its orientation. Then I saw this video. Wow. https://www.youtube.com/watch?v=KZVgKu6v808 https://www.youtube.com/watch?v=KZVgKu6v808 This is from a series of videos called "Engineerguy"
- mrestko 10y ago
  His videos are wonderful.

intrasight 10y ago

Cool. So when can we get our hands on (or should I say hands off) developer kits?

Animats 10y ago

Small radar ICs will be useful for other things. Radar gives you range rate directly. Getting range rate from vision is hard and takes several frame times at least. If you have position and range rate and dump it into a machine learning system, gesture recognition should be straightforward. It's not clear how useful this will be for GUIs in general. It could be useful for interacting with large screen displays, where you may be too far back for touch and a mouse is a pain. A presentation version could be useful. (Presentations from computers still look lame. The audience usually sees the presenter's desktop while they futz with the computer, then enter PowerPoint.)

semi-extrinsic 10y ago
> Presentations from computers still look lame. The audience usually sees the presenter's desktop while they futz with the computer, then enter PowerPoint. 1. Unplug VGA/HDMI cable 2. Do all of the futzing 3. Plug VGA/HDMI cable This is a solved technical problem. That people can't be bothered is the real problem.
- kevhito 10y ago
  Alternative if you present regularly: 1. Enable "extended destkop", rather than "mirror desktop". 2. Select an appropriate wallpaper, or just black.
  - digi_owl 10y ago
    Best i can tell, presenters use mirror so that they can see their own slides on the screen while talking. Frankly i think a seldom talked about Android feature may be a better solution. https://developer.android.com/about/versions/android-4.2.html#SecondaryDisplays https://developer.android.com/about/versions/android-4.2.htm...
    - anc84 10y ago
      Any presentation tool worth using has a presenter mode where on your primary display you see things like the current slide, your notes, a timer, the upcoming slide, etc.

neurotech1 10y ago

They've had "small" radar modules for a while, commonly used as microwave movement detectors[0] Also, here is the manufacturer brochure[1] [0] http://dangerousprototypes.com/blog/2013/08/21/breakout-board-uses-hb100-doppler-motion-sensor/ http://dangerousprototypes.com/blog/2013/08/21/breakout-boar... [1] http://www.agilsense.com/useruploads/files/01_HB%20selection%20list%20ver300.pdf http://www.agilsense.com/useruploads/files/01_HB%20selection...

ChuckMcM 10y ago
The difference being Multi-Antenna Synthetic Aperture RADAR vs single antenna RADAR.
- wyager 10y ago
  Precisely. You need synthetic aperture (or similarly complicated Fourier analysis) with multiple transceive pathways to get a good 3D model of the environment. It's very impressive that this project managed to miniaturize both the radio hardware and the computational hardware required for this. You need to perform FFTs very fast to make this work.

krapht 10y ago

I wish there was more technical detail. I would love to know the performance characteristics of the radar, and what techniques it uses (eg. is it fully coherent, waveform repetition frequency, etc).

trhway 10y ago

so they are using something like 60-66GHz which beside small antenna size advantage makes for the fastest attenuation http://www.propagation.gatech.edu/ECE6390/project/Fall2012/Team09/Team9GeoSatTech_website_FINAL/SatCom%20website/images/GaseousAttenuation.png http://www.propagation.gatech.edu/ECE6390/project/Fall2012/T... . Pretty logical.

jhallenworld 10y ago

Many alarm systems have gunn-diode microwave motion sensors in them. Replace this with high-resolution radar and networking for house-wide whole-body gesture recognition.

Animats 10y ago

If this is available with a few meters of range, it has potential for automatic door control. Star Trek quality door control has been done experimentally [1] but the standard sensors are too dumb. You want range, azimuth and range rate. This is no big deal in a radar; it just costs too much. If someone is moving fast, the door opens sooner, and faster. Moving parallel to the door should not trigger it. (In Japan, automatic sliding doors on street storefronts are common, and false triggers from passers-by are a problem.) With good sensors on both sides, you can open the door further in advance if there's someone on the other side. This allows opaque automatic doors. [1] https://www.youtube.com/watch?v=WhrLetrTSYA https://www.youtube.com/watch?v=WhrLetrTSYA

jdhawk 10y ago
60Ghz radio, 2-3 foot range. Soli's sensors can detect motion at a range of about two to three feet, Schwesig says, so any device you use within that range stands to benefit. http://mashable.com/2015/05/30/google-project-soli-analysis/#I.OrHXCphqqm http://mashable.com/2015/05/30/google-project-soli-analysis/...

internaut 10y ago

I want to combine one of these chips with a magic mirror. If this works, then it is one of Google's greatest projects, the number of applications are huge.

dudifordMann 10y ago

Another great tool for touch-less interfaces. I think the odd aspect of this Google I/O demonstration is the novelty usage of the technology for a smart watch. This tech seems like it is more difficult than it's worth in that form factor, but for devices that must remain sanitary, say in a kitchen, or a medical device in an OR, I like the potential.