5 ms·

I worked with a coder about 10 years ago who had been looking into speech recognition for video games. He came up with some pretty interesting prototypes that h

I worked with a coder about 10 years ago who had been looking into speech recognition for video games. He came up with some pretty interesting prototypes that he ultimately had to throw away. His companies legal department told him speech recognition is a legal minefield with most algorithms patented and some very trigger happy lawyers defending those patents. In the end they went with an inferior solution from one of the big hardware manufacturers who coincidentally were also mentioned by the legal team as a patent holder. So maybe real research is being hampered?

arethuza 16y ago
Some games do have voice control - Rainbow Six: Vegas on the Xbox 360 allows you to give order to your computer controlled teammates.
- kmfrk 16y ago
  Having Microsoft and other platform owners to cover your back as a semi-big game also helps. :)
- dazzawazza 16y ago
  I'd be willing to bet that they use the hardware vendors solution. I'm sure both MS and Sony provide libraries for XBox 2 and PS3 (not sure about Wii).
- Tycho 16y ago
  I remember Rainbow 6 III Black Arrow or something had very usable voice recognition for the solo game. You basically had a few commands like 'regroup,' 'go go go,' 'hold this position' which worked out better than button commands. It seemed to disappear in subsequent Tom Clancy games but glad to hear they brought it back for Vegas. I think the key with speech recognition and natural language processing is to forget the HAL9000/StarTrek 'hello computer' nonsense, and focus on constrained domains augmented by other technologies. Touchpad keyboards are blatantly less usable than hard keyboards but by the time you factor in a) portability and b) auto-completion of words/urls/search-strings, then they become a more than viable alternative. People have incredibly adaptive linguistic skills, developers should harness this more IMO rather than be overly ambitious with NLP. That said I was very impressed with the accuracy of Google's voice search app. But like Jeff Atwood says, it's easier just to search normally.
- zokier 16y ago
  Tom Clancy's Endwar had quite prominent voice recognition interface.
cdavid 16y ago
I don't think this is an issue, at least in academia. The one case where patent has been an issue is speech synthesis, where the PSOLA method has been patented and was the state of the art for quite some time. I am not an expert on speech recognition, but I have done my PhD in a speech lab, so I have at least an idea on the issues. Almost every current speech recognizer is divided into two parts: the acoustical modeling (from audio file to "phonemes"), and language models (from "phonemes" to words). Most people working on acoustical modeling are interested in difficult situations (multi speakers, noisy speech, etc... where performances are nowhere near human in general). The language modeling part has to deal with sparsity: the idea is to model the probability of getting a work w_n given that the previous words were (w_n-1, w_n-2, ...). Given that vocabularies (numbers of possible words) are of the order of millions for large vocabulary recognition, you can imagine that many (most) combinations are never seen in the training data (trillions and more combinations), so you have to somewhat "smooth" the data, remove early "impossible" combinations., etc... There are a lot of heuristics. Also, even though the author is a bit off on the timing, he is right that the basic methods are the same for a long time (statistical, data-driven, HMM for the acoustical model). Sure, we now have (somewhat) speaker-independent models so that you don't have to train the model with your own voice for hours anymore, and language models can handle large vocabularies, but the basics are really the same. A researcher who would have hibernated 20 years ago and just woke up would be able to update in no time. Proposing a new system is difficult, because the current systems are extremely hard to beat: the state of the art requires thousand of hours of training, which means you are almost required to use a lot of programs outside your own lab. For example, almost everyone uses the same software for acoustical modelling (HTK), same labeled data, etc... The published improvements are often tiny (less than one point, e.g. going from 79 % to 80 %), and I actually wonder if those are scientifically significant. Using things like pitch and other non verbal cues (prosody is the actual term used in the literature) is often suggested, but estimating those reliably is extremely difficult. I would guess this will improve because some of languages which are currently funded require prosodic information. Chinese is the obvious example: Chinese has the notion of tones, where the pitch of phonemes may radically change the meaning of the word. On the bright side, I think speech recognition was one of the first field which pushed the idea of data-driven models (at least 3 decades when the first usage of HMM appear), and as such, has been on the forefront of the current explosion of non trivial statistical models trained on big datasets. In that sense, I think it had non trivial contribution outside its own application.

arethuza 16y ago

Some games do have voice control - Rainbow Six: Vegas on the Xbox 360 allows you to give order to your computer controlled teammates.

kmfrk 16y ago
Having Microsoft and other platform owners to cover your back as a semi-big game also helps. :)
dazzawazza 16y ago
I'd be willing to bet that they use the hardware vendors solution. I'm sure both MS and Sony provide libraries for XBox 2 and PS3 (not sure about Wii).
Tycho 16y ago
I remember Rainbow 6 III Black Arrow or something had very usable voice recognition for the solo game. You basically had a few commands like 'regroup,' 'go go go,' 'hold this position' which worked out better than button commands. It seemed to disappear in subsequent Tom Clancy games but glad to hear they brought it back for Vegas. I think the key with speech recognition and natural language processing is to forget the HAL9000/StarTrek 'hello computer' nonsense, and focus on constrained domains augmented by other technologies. Touchpad keyboards are blatantly less usable than hard keyboards but by the time you factor in a) portability and b) auto-completion of words/urls/search-strings, then they become a more than viable alternative. People have incredibly adaptive linguistic skills, developers should harness this more IMO rather than be overly ambitious with NLP. That said I was very impressed with the accuracy of Google's voice search app. But like Jeff Atwood says, it's easier just to search normally.
zokier 16y ago
Tom Clancy's Endwar had quite prominent voice recognition interface.

cdavid 16y ago

I don't think this is an issue, at least in academia. The one case where patent has been an issue is speech synthesis, where the PSOLA method has been patented and was the state of the art for quite some time. I am not an expert on speech recognition, but I have done my PhD in a speech lab, so I have at least an idea on the issues. Almost every current speech recognizer is divided into two parts: the acoustical modeling (from audio file to "phonemes"), and language models (from "phonemes" to words). Most people working on acoustical modeling are interested in difficult situations (multi speakers, noisy speech, etc... where performances are nowhere near human in general). The language modeling part has to deal with sparsity: the idea is to model the probability of getting a work w_n given that the previous words were (w_n-1, w_n-2, ...). Given that vocabularies (numbers of possible words) are of the order of millions for large vocabulary recognition, you can imagine that many (most) combinations are never seen in the training data (trillions and more combinations), so you have to somewhat "smooth" the data, remove early "impossible" combinations., etc... There are a lot of heuristics. Also, even though the author is a bit off on the timing, he is right that the basic methods are the same for a long time (statistical, data-driven, HMM for the acoustical model). Sure, we now have (somewhat) speaker-independent models so that you don't have to train the model with your own voice for hours anymore, and language models can handle large vocabularies, but the basics are really the same. A researcher who would have hibernated 20 years ago and just woke up would be able to update in no time. Proposing a new system is difficult, because the current systems are extremely hard to beat: the state of the art requires thousand of hours of training, which means you are almost required to use a lot of programs outside your own lab. For example, almost everyone uses the same software for acoustical modelling (HTK), same labeled data, etc... The published improvements are often tiny (less than one point, e.g. going from 79 % to 80 %), and I actually wonder if those are scientifically significant. Using things like pitch and other non verbal cues (prosody is the actual term used in the literature) is often suggested, but estimating those reliably is extremely difficult. I would guess this will improve because some of languages which are currently funded require prosodic information. Chinese is the obvious example: Chinese has the notion of tones, where the pitch of phonemes may radically change the meaning of the word. On the bright side, I think speech recognition was one of the first field which pushed the idea of data-driven models (at least 3 decades when the first usage of HMM appear), and as such, has been on the forefront of the current explosion of non trivial statistical models trained on big datasets. In that sense, I think it had non trivial contribution outside its own application.