6 ms·

Gmail Motion April Fools' gag inevitably turned into reality using Kinect

147 points by stevederico 16y ago

incomethax 16y ago
Clickable link to the project website: http://projects.ict.usc.edu/mxr/faast/ http://projects.ict.usc.edu/mxr/faast/ This makes me want to get a kinect to play around with.
Tycho 16y ago
The sad thing is, I saw Google Motion appear in my Gmail toolstrip, but I didn't even think about April fools, I just thought 'oh, great, another half-baked wannabe social media experiment masquerading as a feature from Google (the email and search company)' and didn't bother clicking it.
daimyoyo 16y ago
Has anyone combined the Kinect with iOS yet? That plus the HDMI out from the new iPad to a big screen tv would equal epicness on an unprecedented scale. In fact, I'd actually pay more for that than I did for the iPad itself. Get to work, Apple.
- lukifer 16y ago
  I would bet any amount of money that there are iPad prototypes in R&D that use 2 or 4 front-facing cameras.
  - jrockway 16y ago
    I would bet $10 that there aren't.
    - kgermino 16y ago
      I would bet $100 neither of you have any way to know. :)
  - unsigner 16y ago
    Kinect doesn't work using 2 or 4 front-facing camera and analysing the differences; it projects a IR pattern of dots and analyses its distortion. They say it's more robust and requires much less CPU power.
random42 16y ago
That was really fast. I feel pretty incompetent (which in all likelihood, may not be far from true, despite what my client/managers say). Good job!
troymc 16y ago
I knew Gmail Motion was an April Fools gag --- that's what the link sending me there said, but... It's actually a good idea --- for certain users. In particular, there are many people who use sign language to communicate. For those who sign, what could be more natural than controlling Gmail via sign? And why stop at Gmail? Further reading: http://cad.ca/en/issues/telecommunications.asp http://cad.ca/en/issues/telecommunications.asp
- ernestipark 16y ago
  I'm taking a multimodal user interfaces course, and for our final project my partner and I discussed a gesture based interface for GMail on March 31. 10 hours later I saw GMail Motion. Granted, we weren't thinking quite the gesture set Google had in mind, but similar idea.
edanm 16y ago
Watching the "joke" made me realize something - there's a chance that, in the future, a lot of communication with computers will be done using sign language. After all, most people (rightly) assume that pretty soon, voice recognition will work well enough that you'll be able to dictate everything to your computer, making keyboards partially obsolete (especially for the majority of users). But everyone also correctly thinks that there is one big problem with voice-driven computers - they won't work well with many people in the room at the same time. Imagine a cubicle environment in which everyone needs to talk to their computers. Motion-detection, coupled with sign language, is a pretty logical next step.
- rimantas 16y ago
  I cannot imagine why would I choose to use voice recognition even if I am alone with computer.
  - kabdib 16y ago
    In Netflix, today: "Xbox, pause." (handle phone call). "Xbox, play." For simple commands, it's pretty neat. "Xbox, order pizza and find a movie that doesn't suck" is probably a few years off :-)
    - jrockway 16y ago
      If only there was a small keyboard with a limited set of buttons that could be pressed to command consumer electronics devices. I would call it ... a "remote control".
      - kabdib 16y ago
        On a random remote I just picked up, I see buttons labeled: Menu, Top Menu, Configure, Category, Setup, Option, two Enter buttons, two Mute buttons, several banks of numeric entry 0-9, two channel changers, two volume changers, three different power buttons, two mutes, and four ways to change inputs. I would much rather do this: "Computer, show me some God damned teevee." "Yes, boss." I think it's likely we'll get decent voice-and-intent recognition before the remote control makers are brought, well, under control. Right now your average remote is like flying a fapping 747. [I'm aware of the Apple remote. It seems cool, but I haven't used one.]
motters 16y ago
All joking aside, for some users with disabilities this sort of gesture based interface could be really useful. My prediction is that gesture interfaces are going to be used for some basic things, but that most interactions with computers will remain fairly conventional. If you do a time and motion study, the way people currently interact via keyboards, mice and touch screens is pretty efficient. Even with augmented reality, I expect that users will still be tapping on virtual keyboards.
- lazerwalker 16y ago
  Other than someone with serious finger/hand issues, I'm having a hard time imagining a disability that would prohibit someone from using a mouse/keyboard as input but would leave them able to use full-body gestures.
  - anasol 16y ago
    It is not necessarily a bad idea, and it would not have to be restrained to people with disabilities. The way Google presented the idea was fun but it did make some good sense: it would make us, couch potatoes "have to" move.
defrost 16y ago
Real time hand sign recognition had a published solution in 2005 ( http://portal.acm.org/citation.cfm?id=1107692 http://portal.acm.org/citation.cfm?id=1107692 "an automatic Australian sign language (Auslan) recognition system, which tracks multiple target objects (the face and hands) throughout an image sequence and extracts features for the recognition of sign phrases. ") after about 10 to 15 years of prior work (eg: Hand movement classification using an adaptive fuzzy expert system -https://www.socrates.uwa.edu.au/Pub/PubDetailView.aspx?PublicationID=429191 https://www.socrates.uwa.edu.au/Pub/PubDetailView.aspx?Publi... ) - 1996). It's interesting that this work flowed out of research and development in sheep shearing robotics ( http://school.mech.uwa.edu.au/~jamest/shearmagic/autoshear.html http://school.mech.uwa.edu.au/~jamest/shearmagic/autoshear.h... ) from the late 1970s / early 1980s.
- Kliment 16y ago
  All the tools needed to build this are now built into OpenCV (and likely other similar libs, but OpenCV is what I know) so it's really a matter of a day or two to implement this now. I personally find this amazing. The major difference, I think, between today and 2005 is that today this is doable on your average netbook as someone's weekend project.
  - nitrogen 16y ago
    Is there a recommended OpenCV book or website for the experienced hacker whose career focus has not been computer vision who wants to assemble some of those tools into a larger product?
    - Kliment 16y ago
      Why yes there is. The OpenCV book, from Oreilly. It's excellent for getting to know the library more deeply.
pcanella 16y ago
I wish ThinkGeek would make the Apple Store Playmobil set :(
Joshim5 16y ago
Just as @comex anticipated.... @comex: "I give it 12 hours until someone makes Gmail Motion for real using the Kinect."
hunterp 16y ago
I remember how xenophobic and arrogant the Google organization was in the early days when it was a hot company. Well they are definitely still arrogant, wasting time on a video like this when the technology to actually do it is not really that difficult.

147 points by stevederico 16y ago

Clickable link to the project website: http://projects.ict.usc.edu/mxr/faast/ http://projects.ict.usc.edu/mxr/faast/ This makes me want to get a kinect to play around with.

Tycho 16y ago

The sad thing is, I saw Google Motion appear in my Gmail toolstrip, but I didn't even think about April fools, I just thought 'oh, great, another half-baked wannabe social media experiment masquerading as a feature from Google (the email and search company)' and didn't bother clicking it.

daimyoyo 16y ago

Has anyone combined the Kinect with iOS yet? That plus the HDMI out from the new iPad to a big screen tv would equal epicness on an unprecedented scale. In fact, I'd actually pay more for that than I did for the iPad itself. Get to work, Apple.

lukifer 16y ago
I would bet any amount of money that there are iPad prototypes in R&D that use 2 or 4 front-facing cameras.
- jrockway 16y ago
  I would bet $10 that there aren't.
  - kgermino 16y ago
    I would bet $100 neither of you have any way to know. :)
- unsigner 16y ago
  Kinect doesn't work using 2 or 4 front-facing camera and analysing the differences; it projects a IR pattern of dots and analyses its distortion. They say it's more robust and requires much less CPU power.

random42 16y ago

That was really fast. I feel pretty incompetent (which in all likelihood, may not be far from true, despite what my client/managers say). Good job!

troymc 16y ago

I knew Gmail Motion was an April Fools gag --- that's what the link sending me there said, but... It's actually a good idea --- for certain users. In particular, there are many people who use sign language to communicate. For those who sign, what could be more natural than controlling Gmail via sign? And why stop at Gmail? Further reading: http://cad.ca/en/issues/telecommunications.asp http://cad.ca/en/issues/telecommunications.asp

ernestipark 16y ago
I'm taking a multimodal user interfaces course, and for our final project my partner and I discussed a gesture based interface for GMail on March 31. 10 hours later I saw GMail Motion. Granted, we weren't thinking quite the gesture set Google had in mind, but similar idea.

edanm 16y ago

Watching the "joke" made me realize something - there's a chance that, in the future, a lot of communication with computers will be done using sign language. After all, most people (rightly) assume that pretty soon, voice recognition will work well enough that you'll be able to dictate everything to your computer, making keyboards partially obsolete (especially for the majority of users). But everyone also correctly thinks that there is one big problem with voice-driven computers - they won't work well with many people in the room at the same time. Imagine a cubicle environment in which everyone needs to talk to their computers. Motion-detection, coupled with sign language, is a pretty logical next step.

rimantas 16y ago
I cannot imagine why would I choose to use voice recognition even if I am alone with computer.
- kabdib 16y ago
  In Netflix, today: "Xbox, pause." (handle phone call). "Xbox, play." For simple commands, it's pretty neat. "Xbox, order pizza and find a movie that doesn't suck" is probably a few years off :-)
  - jrockway 16y ago
    If only there was a small keyboard with a limited set of buttons that could be pressed to command consumer electronics devices. I would call it ... a "remote control".
    - kabdib 16y ago
      On a random remote I just picked up, I see buttons labeled: Menu, Top Menu, Configure, Category, Setup, Option, two Enter buttons, two Mute buttons, several banks of numeric entry 0-9, two channel changers, two volume changers, three different power buttons, two mutes, and four ways to change inputs. I would much rather do this: "Computer, show me some God damned teevee." "Yes, boss." I think it's likely we'll get decent voice-and-intent recognition before the remote control makers are brought, well, under control. Right now your average remote is like flying a fapping 747. [I'm aware of the Apple remote. It seems cool, but I haven't used one.]

motters 16y ago

All joking aside, for some users with disabilities this sort of gesture based interface could be really useful. My prediction is that gesture interfaces are going to be used for some basic things, but that most interactions with computers will remain fairly conventional. If you do a time and motion study, the way people currently interact via keyboards, mice and touch screens is pretty efficient. Even with augmented reality, I expect that users will still be tapping on virtual keyboards.

lazerwalker 16y ago
Other than someone with serious finger/hand issues, I'm having a hard time imagining a disability that would prohibit someone from using a mouse/keyboard as input but would leave them able to use full-body gestures.
- anasol 16y ago
  It is not necessarily a bad idea, and it would not have to be restrained to people with disabilities. The way Google presented the idea was fun but it did make some good sense: it would make us, couch potatoes "have to" move.

defrost 16y ago

Real time hand sign recognition had a published solution in 2005 ( http://portal.acm.org/citation.cfm?id=1107692 http://portal.acm.org/citation.cfm?id=1107692 "an automatic Australian sign language (Auslan) recognition system, which tracks multiple target objects (the face and hands) throughout an image sequence and extracts features for the recognition of sign phrases. ") after about 10 to 15 years of prior work (eg: Hand movement classification using an adaptive fuzzy expert system -https://www.socrates.uwa.edu.au/Pub/PubDetailView.aspx?PublicationID=429191 https://www.socrates.uwa.edu.au/Pub/PubDetailView.aspx?Publi... ) - 1996). It's interesting that this work flowed out of research and development in sheep shearing robotics ( http://school.mech.uwa.edu.au/~jamest/shearmagic/autoshear.html http://school.mech.uwa.edu.au/~jamest/shearmagic/autoshear.h... ) from the late 1970s / early 1980s.

Kliment 16y ago
All the tools needed to build this are now built into OpenCV (and likely other similar libs, but OpenCV is what I know) so it's really a matter of a day or two to implement this now. I personally find this amazing. The major difference, I think, between today and 2005 is that today this is doable on your average netbook as someone's weekend project.
- nitrogen 16y ago
  Is there a recommended OpenCV book or website for the experienced hacker whose career focus has not been computer vision who wants to assemble some of those tools into a larger product?
  - Kliment 16y ago
    Why yes there is. The OpenCV book, from Oreilly. It's excellent for getting to know the library more deeply.

pcanella 16y ago

I wish ThinkGeek would make the Apple Store Playmobil set :(

Joshim5 16y ago

Just as @comex anticipated.... @comex: "I give it 12 hours until someone makes Gmail Motion for real using the Kinect."

hunterp 16y ago

I remember how xenophobic and arrogant the Google organization was in the early days when it was a hot company. Well they are definitely still arrogant, wasting time on a video like this when the technology to actually do it is not really that difficult.