3 ms·

This claim seems plausible. The reason seems even simpler than the article. Deep learning requires lots of training data - that data naturally needs to more or

This claim seems plausible. The reason seems even simpler than the article. Deep learning requires lots of training data - that data naturally needs to more or less be "the same"; follow "the same" logic. A long enough time series is going to involve a change in the logic of the real world, a change that the network won't be trained for.

dclowd9901 7y ago
I think this is where the lack of “imagination” that computers currently cannot replicate becomes a huge problem that will set AI and ML back decades more.
- Erlich_Bachman 7y ago
  What are you even referring to? In which of the tasks that ML is currently applied to do they lack "imagination"? GPT-2 generators have plenty of imagination in generating new phrases and meanings... Alphastar has plenty of imagination of making new moves in SCII that even human players haven't come up with yet... Etc..
  - s_Hogg 7y ago
    They're designed to appear that way, it doesn't mean they actually are imagining anything. We need to be very careful about accidental anthropomorphism when it comes to models 0 seeing an analogy between the output in front of you and the human mind can blind us to the reality of what's going on. Which is often more prosaic, even if complicated.
    - Erlich_Bachman 7y ago
      That sounds like you are operating on your own definition which you can apply and shift however you want. This is not what objective science is about. How do you know real humans are not displaying "accidental anthropomorphism"?
s_Hogg 7y ago
Deep Learning doesn't necessarily require a huge amount of data. What DL does is allow you to fit more complex relationships between input and output than would be the case with, say, a bog standard linear model. If a relationship is complex and also clearly defined in your data, then you don't necessarily need much. In general it's true that may not be the case, but that doesn't make "deep learning requires lots of training data" true, only that "the data used for deep learning models is typically noisy on top of representing a complex relationship". It's a semantic difference, but a very important one if we're to avoid going down the road of just mindlessly throwing compute at every problem. And if we do that, we'll just wind up with millions of Rube Goldberg machines instead of actually solving problems. The change in logic of the real world thing is absolutely spot on, though. Over enough time it becomes basically impossible to disentangle effects.
- joe_the_user 7y ago
  Deep Learning doesn't necessarily require a huge amount of data. References? I mean, I know "one shot" and similar approaches but as far as I know, these involve extending a neural network that has been already trained, on massive data, to a little bit more.

dclowd9901 7y ago

I think this is where the lack of “imagination” that computers currently cannot replicate becomes a huge problem that will set AI and ML back decades more.

Erlich_Bachman 7y ago
What are you even referring to? In which of the tasks that ML is currently applied to do they lack "imagination"? GPT-2 generators have plenty of imagination in generating new phrases and meanings... Alphastar has plenty of imagination of making new moves in SCII that even human players haven't come up with yet... Etc..
- s_Hogg 7y ago
  They're designed to appear that way, it doesn't mean they actually are imagining anything. We need to be very careful about accidental anthropomorphism when it comes to models 0 seeing an analogy between the output in front of you and the human mind can blind us to the reality of what's going on. Which is often more prosaic, even if complicated.
  - Erlich_Bachman 7y ago
    That sounds like you are operating on your own definition which you can apply and shift however you want. This is not what objective science is about. How do you know real humans are not displaying "accidental anthropomorphism"?

s_Hogg 7y ago

Deep Learning doesn't necessarily require a huge amount of data. What DL does is allow you to fit more complex relationships between input and output than would be the case with, say, a bog standard linear model. If a relationship is complex and also clearly defined in your data, then you don't necessarily need much. In general it's true that may not be the case, but that doesn't make "deep learning requires lots of training data" true, only that "the data used for deep learning models is typically noisy on top of representing a complex relationship". It's a semantic difference, but a very important one if we're to avoid going down the road of just mindlessly throwing compute at every problem. And if we do that, we'll just wind up with millions of Rube Goldberg machines instead of actually solving problems. The change in logic of the real world thing is absolutely spot on, though. Over enough time it becomes basically impossible to disentangle effects.

joe_the_user 7y ago
Deep Learning doesn't necessarily require a huge amount of data. References? I mean, I know "one shot" and similar approaches but as far as I know, these involve extending a neural network that has been already trained, on massive data, to a little bit more.