6 ms·
The best [speech] recognition engine in the world, and it hears the wrong word more than 6% of the time. Ouch. I would have though the state of the art would b
by danielvf 10y ago
The best [speech] recognition engine in the world, and it hears the wrong word more than 6% of the time. Ouch.
I would have though the state of the art would be better, given anecdotal evidence from friends who write with speech to text programs, and love them.
Perhaps some of this is due to deliberately bad audio quality in the switchboard samples.
- eva1984 10y agoWell, we have to put human into comparison in order to know exactly how impressive 6% metric is. I would guess it might be pretty close.
- rajeshp1986 10y agoI wouldn't blame the audio quality. It is more about different humans have different kinds of accent which makes the same word sound differently. Considering the different accents, I would say 6% error rate is pretty good compared where we started.
- aswanson 10y agoWhy shouldn't the audio quality factor in? Isn't it easier to understand 44.1 Khz 16 bit CD quality audio than 8khz 8 bit pcm (phone data).
- rhizome 10y agoIt might be, but the resource demands of higher fidelity acoustic models slows processing down. 44.1/16 has an order of magnitude greater bitrate than 8/8.
- dharma1 10y agoI guess the point of this particular data set is to check performance on low quality phone audio
- rhizome 10y agoThe base product use case has been to handle phone fidelity for many years. Think: legal dictation, retail digital recording hardware (phones), and medical transcription. Speech-to-text for recordings fed by a Telefunken U-47 is highly niche. :)
- dharma1 10y agoheh. It's not so much because of the microphones that the data sets are THAT narrow band - it's more phone bandwidth limitations. Even the cheapest electret and MEMS mics have pretty good freq response, far beyond 4khz (nyquist 8khz) this data set uses. Now that bandwidth is becoming less of an issue, we will be getting less shitty sounding, wider bandwidth phone audio - https://en.wikipedia.org/wiki/Wideband_audio https://en.wikipedia.org/wiki/Wideband_audio Though if I had it my way, U87 or U47, or hey even SM7B would be mandatory for all speech recordings :)
- justnikos 10y agoEven humans make mistakes: speach -> speech deliberatly -> deliberately I expect these systems to surpass human ability in the next few years.
- JorgeGT 10y agoI wish I only failed to recognize 6% of the words when listening to crappy teleconference calls...
- ams6110 10y agoYou listen on teleconference calls? I like it when people say "Can you repeat that, I was on mute"
- fma 10y agoMistakes are part of real life, just like how you spelled speach, instead of speech.
- escapecharacter 10y agoThe parent comment has a failure rate of 1/38 words, which means a 2.6% failure rate, and this is one of the best commentators in the world, too.
- deleted 10y ago[deleted]
- plafl 10y agoMy bet is that is not really about audio quality, but because in order to understand speech you need to fully understand the language. Try to write down something you hear in a foreign language. As someone who has learnt English as a foreign language I can assure you that at first I wasn't able to even separate between words, it just sounded as a continous stream of sound. Once you learn enough vocabulary you identify some words. Once you identify enough words you are able to fill the gaps using the context. With practice the whole process happens so fast you don't even notice.
- pesenti 10y agoThe human error rate on this task is 4% (https://pdfs.semanticscholar.org/387e/7349b8e31e316c2a7380603d1b6c0b1417c8.pdf https://pdfs.semanticscholar.org/387e/7349b8e31e316c2a738060...). What is amazing is how consistently people overestimate human performance.