4 ms·

Sorry to be a killjoy, but the premise of this article--that Google developed the speech-recognition technology--hurts my feelings (to say the least) and undere

by orijing 16y ago

Sorry to be a killjoy, but the premise of this article--that Google developed the speech-recognition technology--hurts my feelings (to say the least) and underestimates the contributions of the NLP community. Speech recognition, like machine translation, is academic in origin, and much of the work is still carried out in academia. For example, Google did not "invent" machine translation. No, Google Translate is an adapted version of academic systems. Perhaps the phrase tables are sharded and so is the language model, but the general algorithms are the same. Sure, one of Berkeley's NLP grads is working there, but it's basically an adapted version of what's available. They publish papers like "Stupid Backoff" [1], but that makes them as much a contributor as any other member of the NLP community. Speech recognition is the same thing. Google is the company that takes existing research and adapts it. To claim that Google developed the speech recognition technology is to discredit the contributions of everyone else in the NLP community. Google has been generous at funding NLP research at the university level. Do you consider the results of those research "Google"'s? Ultimately, the main difference is that Google has magnitudes more data and the physical capacity to handle that, not that it solved some systems or architectural bottleneck that has been limiting us. Someone once said that all you need is a crappy model and great data to build a good ML-based algorithm... [1] http://acl.ldc.upenn.edu/D/D07/D07-1090.pdf http://acl.ldc.upenn.edu/D/D07/D07-1090.pdf

thezilch 16y ago
Personally, I read it more as Google _applied_ -- developed useful software -- speech-recognition technology that works. I can recall testing countless softwares like Dragon Talk (mentioned in the article) for assisting my writing of school papers, to no avail. These softwares were sluggish and woefully inaccurate -- this may no longer be the case, as I haven't given them a second chance since. I don't think the article disparages past and present work from the NLP community, but you have to expect Google to catch the limelight for the application of such technology. I don't really see anyone else attempting what Google is attempting at scale. Why wouldn't Google be praised (or an "owner") for being a generous benefactor to a community? Would you not consider research at Googleplex (or any of the 100s of satellite campuses) to be Google's?
- orijing 16y ago
  Oh yeah, I agree that Google should be praised for taking research and producing useful applications. I am not disparaging Google in the least. I respect them for bringing NLP to the masses. I meant to dispel the myth that Google invented speech recognition, or that it developed it. The reality is that it is as much of a developer as anyone else in the NLP community.
CWuestefeld 16y ago
Speech recognition, like machine translation, is academic in nature. I don't understand why you're trying to make this an "us vs. them" thing. What is it about academia that makes it so that Google's research can't be considered a part? Does the fact that there are students around somehow make it different? Google is the company that takes existing research and adapts it. I also think you're missing an important differentiator, that between science and technology. The research you're talking about could be summed up as "learning why the world is the way it is"; that's what a scientist does. But the novel things that Google has done are largely technological. They take that understanding that the scientists have given us, and find a way to apply that to the real world. By and large, Google is full of really good engineers, as opposed to scientists.
- orijing 16y ago
  I don't understand why you're trying to make this an "us vs. them" thing. What is it about academia that makes it so that Google's research can't be considered a part? Does the fact that there are students around somehow make it different? Perhaps I was being unclear with "...in nature". I meant that it was academic in origin, and still much of the work is academic. I did not claim that Google has not contributed, but that I was defending the community against a misconception that Google invented/developed something that is partially the work of NLP academics. Re-read my comment please. In particular: The premise of this article--that Google developed the speech-recognition technology--hurts my feelings (to say the least) and underestimates the contributions of the NLP community. They publish papers like "Stupid Backoff" [1], but that makes them as much a contributor as any other member of the NLP community. I am not saying that Google has not contributed to speech. I am arguing that the premise of the article, that Google DEVELOPED the technology, is misleading and insulting. As for science vs. technology: NLP is very systems and technology heavy. If you haven't done much NLP research, you might think that it's very theoretical. No, it's very much about how to stuff a larger language model in memory, how to take an NP-hard decoding problem and produce a linear time approximation algorithm (decoding), how to take an optimization problem of multiple parameters and optimize over them computationally, etc. I agree there is a distinction between science and technology, but to argue that NLP in the academic world is also not "largely technological" is misleading at best.
joe_the_user 16y ago
Having done only the smallest amount of work trying to apply academic research, I have to say that the standard approach through a lot of AI is to develop good ideas about as far as needed to get a few papers done (though I don't know NLP in particular). The work of putting this stuff together into a system which works consistently and at-scale is hard. Basically, it is unfortunate that the standard academia is publishing papers (or pdf files) rather than publishing libraries. With this standard, the academics can't even readily use each others algorithms with this situation. But when academics have this standard, it seems dumb to hear the complaint "uh no, you didn't do anything but apply our ideas..."
- orijing 16y ago
  I am not an expert on speech recognition, but I am somewhat involved with machine translation. For MT, it's very architectural systems issues that limit us, first from trying different models and algorithms, but also in general. How do you know ideas are "good" enough to be publishable unless you do plenty of experiments involving billion (or trillion) word corpora? I have a hard time imagining that research in other fields don't require validation.
  - joe_the_user 16y ago
    I'm not saying that the papers that get published aren't good or valid. "Good enough to publish a paper" is indeed good. It's just that once the paper is published, it becomes a cul-de-sac, a nice little city with no roads leading in or out, etc. Other researchers can only use the result by reproducing the idea by-hand (or at best through crufty Matlab code). Yes, I'm sure the papers I've scanned involved considerable work and data (I worked in computer vision). But that work is often if not generally unavailable to the reader of the paper. The point is that in creating a working system, Google has to do more than extend academic research, even if academic research involved good ideas that had been given some thorough tests in isolation.
- shadowfox 16y ago
  > Basically, it is unfortunate that the standard academia is publishing papers (or pdf files) rather than publishing libraries. With this standard, the academics can't even readily use each others algorithms with this situation. I am not sure code should be the uniform standard for judging (computer) scientific work. It is much harder to review/validate a theory when it is supplied in the form of code than paper. Of course code can serve as an addendum to your work or may be useful as a demonstration of it. (Many academic authors do seem to do this)
hyperbovine 16y ago
The whole point of the article is that the data-driven approach turns out to be a lot more robust that existing academic initiatives. Could anyone else have done it? No. So I think it is fair to say that they developed this technology.
- orijing 16y ago
  That's not the "whole point of the article". The "approaches" are academic in origin; the difference is that Google has more data to train those approaches. Their "approaches" are not different from "existing academic initiatives." Academia often deals with issues like "how to deal with sparsity" of data (i.e. smoothing); at Google, those issues are less important. Having data doesn't not mean having developed the technology.
pkteison 16y ago
Yes, Google is using other people's work. They didn't invent the computer, for example. So what? If their part of it was negligible, somebody else would have given me a useful voice recognition system by now. Why doesn't notepad have a microphone icon on it? If notepad added a microphone icon tomorrow, do you think it would work well? Check your priorities. Delivering a working solution that I can actually use matters a lot more to me than writing a paper.
contextfree 16y ago
This is a problem with reporting (and public perception) of computer software/services in general. I don't think it's widely appreciated that "breakthroughs" by consumer software/services companies generally are the tip of a decades-deep research iceberg.
alextp 16y ago
Actually, google translate is the best academic system there is. They never lost a single nist evaluation (for an example see http://www.itl.nist.gov/iad/mig//tests/mt/2005/doc/mt05eval_official_results_release_20050801_v3.html http://www.itl.nist.gov/iad/mig//tests/mt/2005/doc/mt05eval_... ), as far as I know, and they consistently publish intermediate results and details in their models. If you go by your logic nobody ever writes software, as most of the building blocks were really invented by other people or adapted from other people's inventions. And I say this as an NLP researcher.

by orijing 16y ago

thezilch 16y ago

Personally, I read it more as Google _applied_ -- developed useful software -- speech-recognition technology that works. I can recall testing countless softwares like Dragon Talk (mentioned in the article) for assisting my writing of school papers, to no avail. These softwares were sluggish and woefully inaccurate -- this may no longer be the case, as I haven't given them a second chance since. I don't think the article disparages past and present work from the NLP community, but you have to expect Google to catch the limelight for the application of such technology. I don't really see anyone else attempting what Google is attempting at scale. Why wouldn't Google be praised (or an "owner") for being a generous benefactor to a community? Would you not consider research at Googleplex (or any of the 100s of satellite campuses) to be Google's?

orijing 16y ago
Oh yeah, I agree that Google should be praised for taking research and producing useful applications. I am not disparaging Google in the least. I respect them for bringing NLP to the masses. I meant to dispel the myth that Google invented speech recognition, or that it developed it. The reality is that it is as much of a developer as anyone else in the NLP community.

CWuestefeld 16y ago

Speech recognition, like machine translation, is academic in nature. I don't understand why you're trying to make this an "us vs. them" thing. What is it about academia that makes it so that Google's research can't be considered a part? Does the fact that there are students around somehow make it different? Google is the company that takes existing research and adapts it. I also think you're missing an important differentiator, that between science and technology. The research you're talking about could be summed up as "learning why the world is the way it is"; that's what a scientist does. But the novel things that Google has done are largely technological. They take that understanding that the scientists have given us, and find a way to apply that to the real world. By and large, Google is full of really good engineers, as opposed to scientists.

orijing 16y ago
I don't understand why you're trying to make this an "us vs. them" thing. What is it about academia that makes it so that Google's research can't be considered a part? Does the fact that there are students around somehow make it different? Perhaps I was being unclear with "...in nature". I meant that it was academic in origin, and still much of the work is academic. I did not claim that Google has not contributed, but that I was defending the community against a misconception that Google invented/developed something that is partially the work of NLP academics. Re-read my comment please. In particular: The premise of this article--that Google developed the speech-recognition technology--hurts my feelings (to say the least) and underestimates the contributions of the NLP community. They publish papers like "Stupid Backoff" [1], but that makes them as much a contributor as any other member of the NLP community. I am not saying that Google has not contributed to speech. I am arguing that the premise of the article, that Google DEVELOPED the technology, is misleading and insulting. As for science vs. technology: NLP is very systems and technology heavy. If you haven't done much NLP research, you might think that it's very theoretical. No, it's very much about how to stuff a larger language model in memory, how to take an NP-hard decoding problem and produce a linear time approximation algorithm (decoding), how to take an optimization problem of multiple parameters and optimize over them computationally, etc. I agree there is a distinction between science and technology, but to argue that NLP in the academic world is also not "largely technological" is misleading at best.

joe_the_user 16y ago

Having done only the smallest amount of work trying to apply academic research, I have to say that the standard approach through a lot of AI is to develop good ideas about as far as needed to get a few papers done (though I don't know NLP in particular). The work of putting this stuff together into a system which works consistently and at-scale is hard. Basically, it is unfortunate that the standard academia is publishing papers (or pdf files) rather than publishing libraries. With this standard, the academics can't even readily use each others algorithms with this situation. But when academics have this standard, it seems dumb to hear the complaint "uh no, you didn't do anything but apply our ideas..."

orijing 16y ago
I am not an expert on speech recognition, but I am somewhat involved with machine translation. For MT, it's very architectural systems issues that limit us, first from trying different models and algorithms, but also in general. How do you know ideas are "good" enough to be publishable unless you do plenty of experiments involving billion (or trillion) word corpora? I have a hard time imagining that research in other fields don't require validation.
- joe_the_user 16y ago
  I'm not saying that the papers that get published aren't good or valid. "Good enough to publish a paper" is indeed good. It's just that once the paper is published, it becomes a cul-de-sac, a nice little city with no roads leading in or out, etc. Other researchers can only use the result by reproducing the idea by-hand (or at best through crufty Matlab code). Yes, I'm sure the papers I've scanned involved considerable work and data (I worked in computer vision). But that work is often if not generally unavailable to the reader of the paper. The point is that in creating a working system, Google has to do more than extend academic research, even if academic research involved good ideas that had been given some thorough tests in isolation.
shadowfox 16y ago
> Basically, it is unfortunate that the standard academia is publishing papers (or pdf files) rather than publishing libraries. With this standard, the academics can't even readily use each others algorithms with this situation. I am not sure code should be the uniform standard for judging (computer) scientific work. It is much harder to review/validate a theory when it is supplied in the form of code than paper. Of course code can serve as an addendum to your work or may be useful as a demonstration of it. (Many academic authors do seem to do this)

hyperbovine 16y ago

The whole point of the article is that the data-driven approach turns out to be a lot more robust that existing academic initiatives. Could anyone else have done it? No. So I think it is fair to say that they developed this technology.

orijing 16y ago
That's not the "whole point of the article". The "approaches" are academic in origin; the difference is that Google has more data to train those approaches. Their "approaches" are not different from "existing academic initiatives." Academia often deals with issues like "how to deal with sparsity" of data (i.e. smoothing); at Google, those issues are less important. Having data doesn't not mean having developed the technology.

pkteison 16y ago

Yes, Google is using other people's work. They didn't invent the computer, for example. So what? If their part of it was negligible, somebody else would have given me a useful voice recognition system by now. Why doesn't notepad have a microphone icon on it? If notepad added a microphone icon tomorrow, do you think it would work well? Check your priorities. Delivering a working solution that I can actually use matters a lot more to me than writing a paper.

contextfree 16y ago

This is a problem with reporting (and public perception) of computer software/services in general. I don't think it's widely appreciated that "breakthroughs" by consumer software/services companies generally are the tip of a decades-deep research iceberg.

alextp 16y ago

Actually, google translate is the best academic system there is. They never lost a single nist evaluation (for an example see http://www.itl.nist.gov/iad/mig//tests/mt/2005/doc/mt05eval_official_results_release_20050801_v3.html http://www.itl.nist.gov/iad/mig//tests/mt/2005/doc/mt05eval_... ), as far as I know, and they consistently publish intermediate results and details in their models. If you go by your logic nobody ever writes software, as most of the building blocks were really invented by other people or adapted from other people's inventions. And I say this as an NLP researcher.