5 ms·
To properly translate ASL, you need to pick up a significant amount of grammar from the face (called non-manual markers) as well as body position, 3d capture of
by skrap 10y ago
To properly translate ASL, you need to pick up a significant amount of grammar from the face (called non-manual markers) as well as body position, 3d capture of spatial relationships between signs as they are made, and other features which would be inaccessible to these sensors.
And then there's the fact that ASL isn't serialized English (it's actually unrelated to English at all, and has its roots in the French Sign Language). There's no one-to-one mapping between ASL signs and English words, just as there's no one-to-one mapping between Chinese and French, or any two languages. And ASL has grammar features not seen in any spoken language, to complicate things further.
These sensors could perhaps be used to translate other sign languages, but I'm really skeptical.
- emodendroket 10y agoYeah, even if they could perfectly transcribe the signs you'd have real problems/limitations here I think.
- gugagore 10y agoIt's not really clear what it means to transcribe the signs anyway. There isn't very great notation for signed languages
- Zuider 10y agoI wasn't aware of the existence of any notation at all! The dictionary I have uses photographs, other dictionaries and textbooks use drawings of people speaking SL. The nearest thing to a notation in this case consists of a verbal description of face and hand gestures as they are carried out in real time. I have often wondered if it would be possible to create a notation using the simplest and most abstract rendition of an actual SL speaker's signing. Lines and curves that hold the semantic essence of the movement of face, hand and posture. It would take a native speaker to be able to abstract what is truly essential to meaning, and a community to agree on its elegance and clarity. Off topic, but connected: I discovered from experience that small children can form the SL alphabet with their hands before they acquire the coordination to draw written letters. I found it very useful in teaching my children to read.
- Zuider 10y agoCorrection to my comment above: On taking out my Dictionary of British Sign Language, I find that there is indeed an intricate notation used throughout.
- emodendroket 10y agoDoes it matter? As long as it is consistent and we can reliably translate it (admittedly those are big "ifs") who cares if it's standard?
- gugagore 10y agoYes it matters. I didn't say anything about a standard. I don't mean that "there isn't an agreed upon notation that Deaf Americans use to write ASL", I mean "we don't have a good way represent signs that allows us to search with that representation, apply phonological rules, and synthesize signs from that representation". We have all sorts of representations for spoken speech---a waveform, spectrogram, vocal tract models, IPA--and these all get used in a speech recognition (or synthesis) system. For ASL we have movies, still-image cartoons that depict motion, categories for some parameters like handshape, some linguistic insights, and I'm not sure what else.