5 ms·

Is it just me or does 18% seem like a high error rate - and this is after improvement? I've used technologies (Nuance??) that have significantly lower errors r

by kondro 15y ago

Is it just me or does 18% seem like a high error rate - and this is after improvement? I've used technologies (Nuance??) that have significantly lower errors rates than this, even for systems I have not trained personally. Is there something I'm missing?

huffo 15y ago
Benchmarks have always fascinated me :)
- scq 15y ago
  There are three types of lies: lies, damned lies, and benchmarks.
krmmalik 15y ago
I was going to ask the same thing. It doesn't seem like that significant an improvement but I'm no expert. Also I do wonder if the performance improvements are mostly due to gpu acceleration as opposed to a switch to a different software model.
braindead_in 15y ago
Nuance is speaker dependent. You have to train the system to understand the speaker's voice. This is speaker independent which is much harder.
- mikeash 15y ago
  Nuance has speaker-independent systems as well. If you have an iPhone, try an app like Dragon Dictation or Siri. Neither one requires any training, and will start off with high-quality recognition right out of the box.
romanows 15y ago
The difference in error rates is in large part due to the to the difference between dictated speech and spontaneous, informal conversational speech. Switchboard (http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC97S62 http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=...) is a set of telephone conversations between two people. Speakers tend to say a lot of "ums", abruptly restart an utterance in progress, talk past the telephone handset, etc. Dictated speech, especially when speakers know they're talking to a computer, has less acoustic and linguistic noise.
- danmaz74 15y ago
  It will be very interesting to see how this approach will work with dictated speech. Also let's not forget that the word-level error rate can be reduced by using statistical data about words sequences.

by kondro 15y ago

huffo 15y ago

Benchmarks have always fascinated me :)

scq 15y ago
There are three types of lies: lies, damned lies, and benchmarks.

krmmalik 15y ago

I was going to ask the same thing. It doesn't seem like that significant an improvement but I'm no expert. Also I do wonder if the performance improvements are mostly due to gpu acceleration as opposed to a switch to a different software model.

braindead_in 15y ago

Nuance is speaker dependent. You have to train the system to understand the speaker's voice. This is speaker independent which is much harder.

mikeash 15y ago
Nuance has speaker-independent systems as well. If you have an iPhone, try an app like Dragon Dictation or Siri. Neither one requires any training, and will start off with high-quality recognition right out of the box.

romanows 15y ago

The difference in error rates is in large part due to the to the difference between dictated speech and spontaneous, informal conversational speech. Switchboard (http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC97S62 http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=...) is a set of telephone conversations between two people. Speakers tend to say a lot of "ums", abruptly restart an utterance in progress, talk past the telephone handset, etc. Dictated speech, especially when speakers know they're talking to a computer, has less acoustic and linguistic noise.

danmaz74 15y ago
It will be very interesting to see how this approach will work with dictated speech. Also let's not forget that the word-level error rate can be reduced by using statistical data about words sequences.