37 ms·

Yup https://www.wired.com/2014/06/siri_ai/ https://www.wired.com/2014/06/siri_ai/

hiddencost 10y ago
That's actually really misleading. A system like siri is split up into different parts: -- speech to text -- natural language understanding ( entity recognition, intent classification, entity resolution) -- language generation and speech to text It sounds like what they're saying is that DNNs are used for their speech recognition (speech is split into acoustic and language modeling, and DNN speech models have been state-of-the-art for about 4 years). It's kinda a "so what?" that they were using DNNs for their acoustic models, and nothing to do with the interesting parts of the system. If they were using DNNs for their NLU system back then I'd be kinda surprised.
- future1979 10y ago
  Can you point at a good survey article, blog post, course or book that goes into the NLU aspect? I've watched a bunch of courses on DNN but feel that stuff doesn't apply well to NLU aspects. The speech rec stuff used to be done with HMMs until recently but as you said, DNN is now the way to go for it.
  - hiddencost 10y ago
    (also, sorry, the speech stuff was using HMM+GMM models, and now mostly use HMM+DNN models [so, we replaced the gaussian mixture model with a DNN, but kept the HMM], although some people are moving over to RNN models which doesn't use HMMs (connectionist temporal classification)) This is a great course, about NLP generally: http://cs224d.stanford.edu/syllabus.html http://cs224d.stanford.edu/syllabus.html A typical NLU system might be: an intent classifier to predict intent (e.g., "play music"). This can be pretty much anything (SVM, logistic regression, RNN, I don't care, try stuff and see what works for you) a segmentation model or slot-filling model (so label every word as either junk or "album name" or "song name" or whatever). This requires a sequence model, e.g., HMMs, CRFs, LSTM-RNNs. Here's a nice example: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/TASLP_RNN_SLU_2015.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...

That's actually really misleading. A system like siri is split up into different parts: -- speech to text -- natural language understanding ( entity recognition, intent classification, entity resolution) -- language generation and speech to text It sounds like what they're saying is that DNNs are used for their speech recognition (speech is split into acoustic and language modeling, and DNN speech models have been state-of-the-art for about 4 years). It's kinda a "so what?" that they were using DNNs for their acoustic models, and nothing to do with the interesting parts of the system. If they were using DNNs for their NLU system back then I'd be kinda surprised.

future1979 10y ago
Can you point at a good survey article, blog post, course or book that goes into the NLU aspect? I've watched a bunch of courses on DNN but feel that stuff doesn't apply well to NLU aspects. The speech rec stuff used to be done with HMMs until recently but as you said, DNN is now the way to go for it.
- hiddencost 10y ago
  (also, sorry, the speech stuff was using HMM+GMM models, and now mostly use HMM+DNN models [so, we replaced the gaussian mixture model with a DNN, but kept the HMM], although some people are moving over to RNN models which doesn't use HMMs (connectionist temporal classification)) This is a great course, about NLP generally: http://cs224d.stanford.edu/syllabus.html http://cs224d.stanford.edu/syllabus.html A typical NLU system might be: an intent classifier to predict intent (e.g., "play music"). This can be pretty much anything (SVM, logistic regression, RNN, I don't care, try stuff and see what works for you) a segmentation model or slot-filling model (so label every word as either junk or "album name" or "song name" or whatever). This requires a sequence model, e.g., HMMs, CRFs, LSTM-RNNs. Here's a nice example: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/TASLP_RNN_SLU_2015.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...