3 ms·

I suffer from auditory dyslexia (not officially diagnosed, but it runs in my family and I have all the symptoms, etc). I have a slightly lower than average word

by mabbo 11y ago

I suffer from auditory dyslexia (not officially diagnosed, but it runs in my family and I have all the symptoms, etc). I have a slightly lower than average word recognition rate, especially if I can't see the speakers lips. Yet I get by, because usually the context and what words are expected are enough. This makes me ask two questions: #1- Do systems like this need the court-reported word recognition rate in order to be useful? Or can they compensate for mistakes by using the context? #2- Could we improve these systems by also feeding them video of the speakers lips? Maybe I should go do a masters to figure out the answers.

dave_sullivan 11y ago
That's a good idea, and it probably would help. You're talking about transfer learning or (probably more relevant) multi modal machine learning. You don't need a masters; all the papers are online and everyone has email.
robrenaud 11y ago
Here is a paper that audio and video for speech recognition, and they find that video helps especially in noisy environments. https://www.uni-ulm.de/fileadmin/website_uni_ulm/allgemein/2014_iwsds/iwsds2014_lp_receveur.pdf https://www.uni-ulm.de/fileadmin/website_uni_ulm/allgemein/2...
- mabbo 11y ago
  Awesome! Noisy environments are exactly when seeing the lips is a huge deal for me. I have a friend who has a tendency to absent-mindedly place his hand in front of his mouth. In a quiet office or home, no issue. In a bar? He pressed mute, as far as I'm concerned.
deleted 11y ago
[deleted]
fsam 11y ago
I personally think that context will improve these systems. The current task they are performing is extremely difficult: I think it is analogous to having a listener receive anonymous random phone calls from people they have never heard before, speaking in a random accent, in a difficult/noisy environment, who speak for a few seconds about a random topic including proper nouns that the listener has never heard before and then promptly hang up and the listener is asked for an exact transcription with no mistakes. I don't think it is surprising that the error rates produced by Mechanical Turk workers seem high for some of these tasks, and it actually seems like a big accomplishment that a speech recognition system can do nearly as well under such difficult conditions. Context about the speaker or the current subject of conversation would clearly help, as would visual cues, and the ability to ask the user to repeat or clarify something.

by mabbo 11y ago

dave_sullivan 11y ago

That's a good idea, and it probably would help. You're talking about transfer learning or (probably more relevant) multi modal machine learning. You don't need a masters; all the papers are online and everyone has email.

robrenaud 11y ago

Here is a paper that audio and video for speech recognition, and they find that video helps especially in noisy environments. https://www.uni-ulm.de/fileadmin/website_uni_ulm/allgemein/2014_iwsds/iwsds2014_lp_receveur.pdf https://www.uni-ulm.de/fileadmin/website_uni_ulm/allgemein/2...

mabbo 11y ago
Awesome! Noisy environments are exactly when seeing the lips is a huge deal for me. I have a friend who has a tendency to absent-mindedly place his hand in front of his mouth. In a quiet office or home, no issue. In a bar? He pressed mute, as far as I'm concerned.

deleted 11y ago

[deleted]

fsam 11y ago

I personally think that context will improve these systems. The current task they are performing is extremely difficult: I think it is analogous to having a listener receive anonymous random phone calls from people they have never heard before, speaking in a random accent, in a difficult/noisy environment, who speak for a few seconds about a random topic including proper nouns that the listener has never heard before and then promptly hang up and the listener is asked for an exact transcription with no mistakes. I don't think it is surprising that the error rates produced by Mechanical Turk workers seem high for some of these tasks, and it actually seems like a big accomplishment that a speech recognition system can do nearly as well under such difficult conditions. Context about the speaker or the current subject of conversation would clearly help, as would visual cues, and the ability to ask the user to repeat or clarify something.