4 ms·

The main problem with using stat models or approaches such as deep learning is not that we are unable to do it. Though possibly trivial, the real problem is we

The main problem with using stat models or approaches such as deep learning is not that we are unable to do it. Though possibly trivial, the real problem is we are unable to understand how or why it works which can lead to unintended consequences or lack of ability to support/continue further development (aside from not being able to leverage the new fundamental understanding and apply it to related fields). Imagine if we left our understanding and working model of physics completely up to statistical and deep learning models. We would not be able to reliably design anything or model simulations ect. The reason we have not figured out NLP is not because we are incapable, it’s because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it (even though they could never encounter any other written concept without it)

why-el 5y ago
I think this is the crux of the issue. People like Chomsky argue the same way, that is, sure a statistical model can mimic a phenomenon to an increasingly likely degree, but the question he concerns himself with is "how and why it [language] works that way", not that I can somehow approach it. His example scenario of a filming bees and statistically re-engineer their dance is poignant: sure you get impressive results, but you will never understand why they dance! Once this is clear, the discussion becomes healthier: linguists continue to try and understand fundamentals; ML NLP continues to try to get impressive, possibly lucrative results. Logical research can borrow from ML results, or re-use them, and so forth, but I agree with Chomsky that it can't be a shot in the dark always, because once you deploy it and pass it some tough decisions to make, well, what then? I was pleased to see at least one idea, the recursive structures found in his theories, which is increasingly being accepted by people like Hinton, so that's great.
- visarga 5y ago
  > but you will never understand why they dance! You can't understand why without considering the environment, the problem is that it's too expensive to train AI agents in reality and simulated environments are too simplistic. But if such a simulated environment is available, agents can learn general skills. > Deep mind: Generally capable agents emerge from open-ended play https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e...
- visarga 5y ago
  > but you will never understand why they dance! You can't understand why without considering the environment, the problem is that it's too expensive to train AI agents in reality and simulated environments are too simplistic. But if such a simulated environment is available, agents can learn general skills. > Deep mind: Generally capable agents emerge from open-ended play https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e... In this kind of setup the agents know why they act like they do.
danielmarkbruce 5y ago
I'm not a physicist or mechanical engineer and this is a genuine question - why would we not be able to reliably design anything if we only had statistical models? Empirical formulas are used in various places in science/engineering and seem to work well (with exceptions). As an example - would we not arrive at something like f = ma using statistics and then use it without really knowing why (or maybe all the implications of) f = ma?
- ivalm 5y ago
  I think the main issue is interpolation vs extrapolation. Statistical models work well within the domain where data is available but have trouble generalizing to where data is unavailable. The very cool thing with physics is that theory was able to make a lot of true but wild predictions that only much later were backed up with experiment. Such powerful theories are very useful for finding interesting phenomena to study/exploit
  - danielmarkbruce 5y ago
    I'm saying "does it matter how we arrive at a model?".
sdenton4 5y ago
/because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it/ Well, there was a good fifty plus years of linguists without access to good statistical methods who also failed to solve the problems as well... And they also didn't come up with the kinda functional answers that we now have from statistical methods, which are now able to do crazy things like translate reasonably well between nearly arbitrary language pairs. Your statement sounds like a theoretician's sour grapes...
- suifbwish 5y ago
  50 plus years was before the modern computing era. You mean they couldn’t figure it out on paper.
visarga 5y ago
> unable to understand how or why it works We do understand how and why it works to a certain degree (gradient based input to output approximation). But if you want the meaning of the third neuron in the 10th layer, then yes... But in the same sense we don't understand thermodynamics because we don't account for each particle, just a statistical aggregate of them. And on the other hand, we don't understand much about how people work or how we are motivated, even though we have first and third person perspectives. Psychology seems to be no further advanced than AI. And yet we work with what we have.
sgt101 5y ago
>The reason we have not figured out NLP is not because we are incapable, it’s because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it (even though they could never encounter any other written concept without it) Lots of brilliant people have looked, but the issue at the moment is that the last set of questions and challenges was surprisingly blown away by statistical approaches and we are waiting to find the ceiling on those approaches before being able to formulate the right questions for the next round of deep thinking I suppose. The Winnograd schema were kinda meant to prompt that....
Isinlor 5y ago
We can understand how and why neural networks work. The issue is that obtaining this understanding requires a lot of effort. OpenAI invested in trying to understand how a simple CNN works. You can read their results here: https://distill.pub/2020/circuits/ https://distill.pub/2020/circuits/ > In the original narrative of deep learning, each neuron builds progressively more abstract, meaningful features by composing features in the preceding layer. In recent years, there’s been some skepticism of this view, but what happens if you take it really seriously? > InceptionV1 is a classic vision model with around 10,000 unique neurons — a large number, but still on a scale that a group effort could attack. What if you simply go through the model, neuron by neuron, trying to understand each one and the connections between them? The circuits collaboration aims to find out.
- suifbwish 5y ago
  Thank you for sharing that. Definitely going to read

why-el 5y ago

I think this is the crux of the issue. People like Chomsky argue the same way, that is, sure a statistical model can mimic a phenomenon to an increasingly likely degree, but the question he concerns himself with is "how and why it [language] works that way", not that I can somehow approach it. His example scenario of a filming bees and statistically re-engineer their dance is poignant: sure you get impressive results, but you will never understand why they dance! Once this is clear, the discussion becomes healthier: linguists continue to try and understand fundamentals; ML NLP continues to try to get impressive, possibly lucrative results. Logical research can borrow from ML results, or re-use them, and so forth, but I agree with Chomsky that it can't be a shot in the dark always, because once you deploy it and pass it some tough decisions to make, well, what then? I was pleased to see at least one idea, the recursive structures found in his theories, which is increasingly being accepted by people like Hinton, so that's great.

visarga 5y ago
> but you will never understand why they dance! You can't understand why without considering the environment, the problem is that it's too expensive to train AI agents in reality and simulated environments are too simplistic. But if such a simulated environment is available, agents can learn general skills. > Deep mind: Generally capable agents emerge from open-ended play https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e...
visarga 5y ago
> but you will never understand why they dance! You can't understand why without considering the environment, the problem is that it's too expensive to train AI agents in reality and simulated environments are too simplistic. But if such a simulated environment is available, agents can learn general skills. > Deep mind: Generally capable agents emerge from open-ended play https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e... In this kind of setup the agents know why they act like they do.

danielmarkbruce 5y ago

I'm not a physicist or mechanical engineer and this is a genuine question - why would we not be able to reliably design anything if we only had statistical models? Empirical formulas are used in various places in science/engineering and seem to work well (with exceptions). As an example - would we not arrive at something like f = ma using statistics and then use it without really knowing why (or maybe all the implications of) f = ma?

ivalm 5y ago
I think the main issue is interpolation vs extrapolation. Statistical models work well within the domain where data is available but have trouble generalizing to where data is unavailable. The very cool thing with physics is that theory was able to make a lot of true but wild predictions that only much later were backed up with experiment. Such powerful theories are very useful for finding interesting phenomena to study/exploit
- danielmarkbruce 5y ago
  I'm saying "does it matter how we arrive at a model?".

sdenton4 5y ago

/because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it/ Well, there was a good fifty plus years of linguists without access to good statistical methods who also failed to solve the problems as well... And they also didn't come up with the kinda functional answers that we now have from statistical methods, which are now able to do crazy things like translate reasonably well between nearly arbitrary language pairs. Your statement sounds like a theoretician's sour grapes...

suifbwish 5y ago
50 plus years was before the modern computing era. You mean they couldn’t figure it out on paper.

visarga 5y ago

> unable to understand how or why it works We do understand how and why it works to a certain degree (gradient based input to output approximation). But if you want the meaning of the third neuron in the 10th layer, then yes... But in the same sense we don't understand thermodynamics because we don't account for each particle, just a statistical aggregate of them. And on the other hand, we don't understand much about how people work or how we are motivated, even though we have first and third person perspectives. Psychology seems to be no further advanced than AI. And yet we work with what we have.

sgt101 5y ago

>The reason we have not figured out NLP is not because we are incapable, it’s because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it (even though they could never encounter any other written concept without it) Lots of brilliant people have looked, but the issue at the moment is that the last set of questions and challenges was surprisingly blown away by statistical approaches and we are waiting to find the ceiling on those approaches before being able to formulate the right questions for the next round of deep thinking I suppose. The Winnograd schema were kinda meant to prompt that....

Isinlor 5y ago

We can understand how and why neural networks work. The issue is that obtaining this understanding requires a lot of effort. OpenAI invested in trying to understand how a simple CNN works. You can read their results here: https://distill.pub/2020/circuits/ https://distill.pub/2020/circuits/ > In the original narrative of deep learning, each neuron builds progressively more abstract, meaningful features by composing features in the preceding layer. In recent years, there’s been some skepticism of this view, but what happens if you take it really seriously? > InceptionV1 is a classic vision model with around 10,000 unique neurons — a large number, but still on a scale that a group effort could attack. What if you simply go through the model, neuron by neuron, trying to understand each one and the connections between them? The circuits collaboration aims to find out.

suifbwish 5y ago
Thank you for sharing that. Definitely going to read