4 ms·

Very cool. Unfortunately, it looks bulky. Disney's Touche demonstrated how the capacitive profile across frequencies could detect hand gestures. Ideally a simp

Very cool. Unfortunately, it looks bulky. Disney's Touche demonstrated how the capacitive profile across frequencies could detect hand gestures. Ideally a simple wristband, coupled with a strong predictive model could do the same thing.

zachrose 14y ago
It seems the hard part about this would be mapping kinematics to sign language, hopefully this is decoupled from actually capturing the positions and motions.
- dsrguru 14y ago
  Even harder is translating from the sign language to a spoken language. Machine translation is a very much unsolved problem, and sign languages are no more similar grammatically to spoken languages than spoken languages are to spoken languages.
  - electromagnetic 14y ago
    Even worse a deaf person can't hear a typo when they say fuck pudding instead of fig pudding.
    - morsch 14y ago
      This is a fun thing to think about. Typos: various effects caused by the interface used to write down language (so, broadly comparable to ink blotches), genuis, betwween, [2] Mis-spelling: caused by spelling a word different from a standard[0], whole → hole Mis-speaking/hearings: caused by not producing or processing a word according to standard, or commonly due to losses in transmission, cogenital defect → <traffic noise>genital defect (not sure why that was the first thing I came up with, but I'm sticking with it) Mis-understandings in ASL: I assume these happen when the movement for a certain sign is similar enough that it can be performed or read improperly. I guess ASL was designed/evolved in a way that minimizes this, e.g. by maximizing the contrast for for common neighbours (otherwise efficiency will drive in the other direction). What's fun about this is that these three classes of errors are almost direct analogues in their respective domains, but they probably lead to very different kinds of errors. Typos are basically arbitrary, but "comprehensible" if you know QWERTY. Mis-spelling and mis-hearings both result from the deviation from a standard or from transmission losses[1], and are often easily comprehensible. I assume the same is true for mis-understandings in ASL for people who know ASL -- but the character of being comprehensible gets lost in translation. YouTube sometimes offers the combination of speech recognition plus machine translation. That's a close equivalent to machine translating sign language: audio/video recognition followed by machine translation. I don't think I have ever seen it work on a YouTube video, not even close. And I'm guessing ASL translation is more difficult: transcribing audio is probably easier than recognizing signs, and for audio there is an obvious target domain for transcription (ie. written English/phonetic transcription); whereas I'm not sure if there is any standard way to transcribe ASL, is there a "written ASL"? Come to think of it, there has to be, at least for "meta" tasks like dicussing or teaching ASL in printed form. ... okay so what was I going to do half an hour ago? [0] but often in line with intuitive phonetic rules; you might go as far as calling standard English spelling another unrelated scheme ;) [1] arguably betwween and genuis are also transmission losses; a more extreme example for interface transmission losses (same source and target, different interface) are showcased on pages like damn you autocorrect [2] http://bash.org/?5300 http://bash.org/?5300
  - morsch 14y ago
    Well, we don't have machine translation that does anything resembling natural language. But what we have is a lot better than nothing!

zachrose 14y ago

It seems the hard part about this would be mapping kinematics to sign language, hopefully this is decoupled from actually capturing the positions and motions.

dsrguru 14y ago
Even harder is translating from the sign language to a spoken language. Machine translation is a very much unsolved problem, and sign languages are no more similar grammatically to spoken languages than spoken languages are to spoken languages.
- electromagnetic 14y ago
  Even worse a deaf person can't hear a typo when they say fuck pudding instead of fig pudding.
  - morsch 14y ago
    This is a fun thing to think about. Typos: various effects caused by the interface used to write down language (so, broadly comparable to ink blotches), genuis, betwween, [2] Mis-spelling: caused by spelling a word different from a standard[0], whole → hole Mis-speaking/hearings: caused by not producing or processing a word according to standard, or commonly due to losses in transmission, cogenital defect → <traffic noise>genital defect (not sure why that was the first thing I came up with, but I'm sticking with it) Mis-understandings in ASL: I assume these happen when the movement for a certain sign is similar enough that it can be performed or read improperly. I guess ASL was designed/evolved in a way that minimizes this, e.g. by maximizing the contrast for for common neighbours (otherwise efficiency will drive in the other direction). What's fun about this is that these three classes of errors are almost direct analogues in their respective domains, but they probably lead to very different kinds of errors. Typos are basically arbitrary, but "comprehensible" if you know QWERTY. Mis-spelling and mis-hearings both result from the deviation from a standard or from transmission losses[1], and are often easily comprehensible. I assume the same is true for mis-understandings in ASL for people who know ASL -- but the character of being comprehensible gets lost in translation. YouTube sometimes offers the combination of speech recognition plus machine translation. That's a close equivalent to machine translating sign language: audio/video recognition followed by machine translation. I don't think I have ever seen it work on a YouTube video, not even close. And I'm guessing ASL translation is more difficult: transcribing audio is probably easier than recognizing signs, and for audio there is an obvious target domain for transcription (ie. written English/phonetic transcription); whereas I'm not sure if there is any standard way to transcribe ASL, is there a "written ASL"? Come to think of it, there has to be, at least for "meta" tasks like dicussing or teaching ASL in printed form. ... okay so what was I going to do half an hour ago? [0] but often in line with intuitive phonetic rules; you might go as far as calling standard English spelling another unrelated scheme ;) [1] arguably betwween and genuis are also transmission losses; a more extreme example for interface transmission losses (same source and target, different interface) are showcased on pages like damn you autocorrect [2] http://bash.org/?5300 http://bash.org/?5300
- morsch 14y ago
  Well, we don't have machine translation that does anything resembling natural language. But what we have is a lot better than nothing!