3 ms·

>About a year later, Google had reduced Android’s voice recognition error rate by an astounding 25 percent. lol. This is the grand payoff?

deong 12y ago
Considering that for the preceding 25 years or so, progress in the state of the art had been annual reductions of far less than 1%, it's a pretty big deal.
- astazangasta 12y ago
  From an academic perspective, maybe. In terms of practical use of machine methods, not much. Machine learning is largely hype. I pity the army of PhDs they must have building training sets.
  - sp332 12y ago
    Have you tried Google's voice recognition? It's been fantastic for me.
sp332 12y ago
It was already as good as a bunch of PhD's could make it after working on it for years. Removing 25% of the remaining errors after that is a grand payoff!
DanBC 12y ago
There's a bunch of stuff that gets optimised. You start with big gains, and those get smaller and smaller until you're spending a lot of effort to get 0.5% increase in performance. See, for example, compressing the ENWIK8 in the Hutter Prize. http://prize.hutter1.net/ http://prize.hutter1.net/ If you can compress this approx 100 MB file to less than approx 16 MB (including the decompressor) you win cash. That website shows small decreases in filesize over a few years. Alexander Rhatushnyak 23.May 2009 6.27 | 1614€ Marcus Hutter Alexander Rhatushnyak 14.May 2007 6.07 | 1732€ Marcus Hutter Alexander Rhatushnyak 25.Sep.2006 5.86 | 3416€ Marcus Hutter Matt Mahoney 24.Mar.2006 5.46 | pre-prize - The compression ratio goes from 5.46 to 5.86 to 6.07 to 6.27.
kylebgorman 12y ago
While I'm not sure, I assume the author is referring to "error reduction". Error reduction is defined as def ER(accuracy_before, accuracy_after): delta = accuracy_after - accuracy_before return delta / (1. - accuracy_before) (It is often written as a percentage.) Note that ER(.5, .75) = ER(.98, .99) = .5. It is for this reason I dislike it. Without knowing what performance before and after was like, it's hard to tell whether this was an incremental improvement to an already impressive system or the innovation that made a system useable. (It strikes me as unlikely that the author means to say that Google reduced the Android recognizer's word error rate [WER] by 25%, since WERs on many publicly available databases used for evaluation purposes were already well below 25% before the deep learning revolution. But, it is not impossible that Google's test set was particularly hard and the pre-deep-learning Android recognizer was unimpressive. Caveat: I know, I know, WER isn't really "accuracy".)

deong 12y ago

Considering that for the preceding 25 years or so, progress in the state of the art had been annual reductions of far less than 1%, it's a pretty big deal.

astazangasta 12y ago
From an academic perspective, maybe. In terms of practical use of machine methods, not much. Machine learning is largely hype. I pity the army of PhDs they must have building training sets.
- sp332 12y ago
  Have you tried Google's voice recognition? It's been fantastic for me.

sp332 12y ago

It was already as good as a bunch of PhD's could make it after working on it for years. Removing 25% of the remaining errors after that is a grand payoff!

DanBC 12y ago

There's a bunch of stuff that gets optimised. You start with big gains, and those get smaller and smaller until you're spending a lot of effort to get 0.5% increase in performance. See, for example, compressing the ENWIK8 in the Hutter Prize. http://prize.hutter1.net/ http://prize.hutter1.net/ If you can compress this approx 100 MB file to less than approx 16 MB (including the decompressor) you win cash. That website shows small decreases in filesize over a few years. Alexander Rhatushnyak 23.May 2009 6.27 | 1614€ Marcus Hutter Alexander Rhatushnyak 14.May 2007 6.07 | 1732€ Marcus Hutter Alexander Rhatushnyak 25.Sep.2006 5.86 | 3416€ Marcus Hutter Matt Mahoney 24.Mar.2006 5.46 | pre-prize - The compression ratio goes from 5.46 to 5.86 to 6.07 to 6.27.

kylebgorman 12y ago

While I'm not sure, I assume the author is referring to "error reduction". Error reduction is defined as def ER(accuracy_before, accuracy_after): delta = accuracy_after - accuracy_before return delta / (1. - accuracy_before) (It is often written as a percentage.) Note that ER(.5, .75) = ER(.98, .99) = .5. It is for this reason I dislike it. Without knowing what performance before and after was like, it's hard to tell whether this was an incremental improvement to an already impressive system or the innovation that made a system useable. (It strikes me as unlikely that the author means to say that Google reduced the Android recognizer's word error rate [WER] by 25%, since WERs on many publicly available databases used for evaluation purposes were already well below 25% before the deep learning revolution. But, it is not impossible that Google's test set was particularly hard and the pre-deep-learning Android recognizer was unimpressive. Caveat: I know, I know, WER isn't really "accuracy".)