4 ms·

can someone give me some real world business need where I can apply RNN and this type of knowledge?

dennybritz 11y ago
Author here. RNNs are used in the same places as most other Machine Learning models and may replace some of the older models in the future. What these models do is typically not visible to the end-customer (unless you sell developer tools), but it's an important part of everyday technology. Just take Siri for example. Under the hood Siri uses almost all of the application mentioned in the post: Speech recognition, language modeling and (I think?) machine translation. You can find parts of this in a lot of products that you use every day. In terms of business ROI, RNNs may not only yield better results for the end-user, but may also lead to lower costs due to their simplicity. You may be able to replace a very complex hand-engineered system with a RNN that's easier and cheaper to maintain.
- dxbydt 11y ago
  May yield... May lead...May replace. That's a lot of maybe. I submit that from purely a business ROI pov, you are far better off building a 'very complex hand engineered system' as you put it, since that's the natural outcome of hiring say a bunch of rails/python devs at pennies on the dollar on some offshoot dev portal. Building a nicely tuned scalable RNN model in the industry requires a team with 100x intellectual capabilities for which you will pay 100x, and there may not be a mature business case for that yet.Though I agree much of these skills are being commoditized rapidly.
  - trentnelson 11y ago
    I agree, I don't think we're there yet. I think the tipping point will be when two key things converge: a) when data (particularly historically-oriented, time series-type data) becomes as accessible and as commoditized as `npm install <foo>` (e.g. `datawiz install <a-PB-sized-data-set-of-everything-that's-ever-happened-in-this-domain-ever>`), and b) the realization that software engineers and data scientists work best when paired together; they're symbiotic roles, not competitive/opposing (e.g. think an F-14 pilot and RIO). I wish every keystroke, every OS, every program, console, TTY, GUI, errno, ssh session, Window, RDP frame, TCP/IP packet... everything I ever did was logged and timestamped. Imagine pairing that level of data with an RNN and a feedback loop that could self-evaluate predictions. (And then imagine if that could be anonymized and publicized, such that in 50-100 years from now, new developers could get a head start with a "friendly AI" that nurses them from "well, that segfaulted" to "end-to-end enterprise app implemented from scratch" over the course of their career.)
  - argonaut 11y ago
    Seems like a bit of a non sequitur. Nothing is guaranteed in life. That being said, you've got this backwards. The natural outcome of hiring a bunch of rails/python devs to fine tune a machine learning / translation / recommendation system is that you get hundreds of thousands of lines of code that run slowly AND don't work. The entire premise of "deep" learning is that the system is a black box - features are learned by the black box. And you typically use pre-rolled fast GPU implementations. Most importantly, very little domain specific knowledge is needed. In fact, getting that hand-engineered system is going to be more complex, more costly, and it's going to require people with more domain expertise.
- curiousjorge 11y ago
  what are some examples of 'complex hand-engineered systems'?

Author here. RNNs are used in the same places as most other Machine Learning models and may replace some of the older models in the future. What these models do is typically not visible to the end-customer (unless you sell developer tools), but it's an important part of everyday technology. Just take Siri for example. Under the hood Siri uses almost all of the application mentioned in the post: Speech recognition, language modeling and (I think?) machine translation. You can find parts of this in a lot of products that you use every day. In terms of business ROI, RNNs may not only yield better results for the end-user, but may also lead to lower costs due to their simplicity. You may be able to replace a very complex hand-engineered system with a RNN that's easier and cheaper to maintain.

dxbydt 11y ago
May yield... May lead...May replace. That's a lot of maybe. I submit that from purely a business ROI pov, you are far better off building a 'very complex hand engineered system' as you put it, since that's the natural outcome of hiring say a bunch of rails/python devs at pennies on the dollar on some offshoot dev portal. Building a nicely tuned scalable RNN model in the industry requires a team with 100x intellectual capabilities for which you will pay 100x, and there may not be a mature business case for that yet.Though I agree much of these skills are being commoditized rapidly.
- trentnelson 11y ago
  I agree, I don't think we're there yet. I think the tipping point will be when two key things converge: a) when data (particularly historically-oriented, time series-type data) becomes as accessible and as commoditized as `npm install <foo>` (e.g. `datawiz install <a-PB-sized-data-set-of-everything-that's-ever-happened-in-this-domain-ever>`), and b) the realization that software engineers and data scientists work best when paired together; they're symbiotic roles, not competitive/opposing (e.g. think an F-14 pilot and RIO). I wish every keystroke, every OS, every program, console, TTY, GUI, errno, ssh session, Window, RDP frame, TCP/IP packet... everything I ever did was logged and timestamped. Imagine pairing that level of data with an RNN and a feedback loop that could self-evaluate predictions. (And then imagine if that could be anonymized and publicized, such that in 50-100 years from now, new developers could get a head start with a "friendly AI" that nurses them from "well, that segfaulted" to "end-to-end enterprise app implemented from scratch" over the course of their career.)
- argonaut 11y ago
  Seems like a bit of a non sequitur. Nothing is guaranteed in life. That being said, you've got this backwards. The natural outcome of hiring a bunch of rails/python devs to fine tune a machine learning / translation / recommendation system is that you get hundreds of thousands of lines of code that run slowly AND don't work. The entire premise of "deep" learning is that the system is a black box - features are learned by the black box. And you typically use pre-rolled fast GPU implementations. Most importantly, very little domain specific knowledge is needed. In fact, getting that hand-engineered system is going to be more complex, more costly, and it's going to require people with more domain expertise.
curiousjorge 11y ago
what are some examples of 'complex hand-engineered systems'?