6 ms·

Reading this paper reminds me how important it is for AI to continue to evolve it’s core algorithms. Deep learning models are effectively pattern matching mach

Reading this paper reminds me how important it is for AI to continue to evolve it’s core algorithms. Deep learning models are effectively pattern matching machines that cannot separate causality from correlation. As we throw more compute and $$$ at deep learning models we will experience diminishing returns in performance because of this. For us to achieve AGI[0], the holy grail of AI, we will need to develop algorithms that can recognize causality somehow. Judea Pearl’s “The Book of Why”[1] does a great job articulating why this is important. Deep learning is a big leap forward and we’re only beginning to see its impacts, but it’s not sufficient to achieve AI’s most ambitious goals. [0] https://en.m.wikipedia.org/wiki/Artificial_general_intelligence https://en.m.wikipedia.org/wiki/Artificial_general_intellige... [1] https://www.amazon.com/Book-Why-Science-Cause-Effect/dp/046509760X https://www.amazon.com/Book-Why-Science-Cause-Effect/dp/0465...

PeterisP 6y ago
The limitation here is not necessarily in the algorithms but in the data. There's no reason to suppose that it's even possible to recognize causality and separate it from correlation purely from static observations - among others, Judea Pearl has written a bunch about it, this year's ACL best theme paper (Bender&Koller, Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data) was on a very similar topic, IIRC there's neuroscience research that suggests that mammals can't "learn to see" functionally if they only get visual input that's not caused by their own acts, etc. It's highly likely that no algorithm or mind can learn what you propose solely from the data (and the type of data) that was available to train GPT. It seems that recognizing causality requires intervention, not just observation, it requires active agents, not just passive specators. It requires "play" and experimentation. Learning language meaning requires at least some grounding and alignment with reality. The problems of causality illustrate the limitations and necessity of integrating models with the environment, they do not indicate any fundamental limitations with model structure itself; perhaps deep learning isn't sufficient for learning these structures, and perhaps it is, as far as I know we don't have any good evidence to justify assuming one way or the other. On the other hand, if/when some artificial agent has learned the core concepts of causality and language grounding in some small environment - using whatever algorithms are required for that - then it seems plausible that a GPT-like system could be sufficient to provide the mapping to extend these concepts to the whole range of things that we talk about; once the agent understands (according to whatever reasonable nontrivial definition of "understands") the concept of causality for some actions, it can learn the causality of all the other actions which are described if you read all the written text in the world.
- peterthehacker 6y ago
  This is a great point!
- libraryofbabel 6y ago
  So goes the old teachers’ proverb: “Tell me and I forget, show me and I may remember, involve me and I understand.”
- marmaduke 6y ago
  This is a good argument for why AGI is more likely to come from AI gym environments where it learns to control an agent in an environment. It isn’t presented with a static dataset but interacts.
chillacy 6y ago
> cannot separate causality from correlation Neither can most humans without rigorous experimentation. Deep learning today seems to emulate only system 1. It just turns out that much of what expert humans do can be attributed to finely tuned system-1 intuition. To achieve AGI we'd need a system-2. (nomenclature for system-1 and system-2 taken from Kahneman and Tversky's work on human cognition)

PeterisP 6y ago

The limitation here is not necessarily in the algorithms but in the data. There's no reason to suppose that it's even possible to recognize causality and separate it from correlation purely from static observations - among others, Judea Pearl has written a bunch about it, this year's ACL best theme paper (Bender&Koller, Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data) was on a very similar topic, IIRC there's neuroscience research that suggests that mammals can't "learn to see" functionally if they only get visual input that's not caused by their own acts, etc. It's highly likely that no algorithm or mind can learn what you propose solely from the data (and the type of data) that was available to train GPT. It seems that recognizing causality requires intervention, not just observation, it requires active agents, not just passive specators. It requires "play" and experimentation. Learning language meaning requires at least some grounding and alignment with reality. The problems of causality illustrate the limitations and necessity of integrating models with the environment, they do not indicate any fundamental limitations with model structure itself; perhaps deep learning isn't sufficient for learning these structures, and perhaps it is, as far as I know we don't have any good evidence to justify assuming one way or the other. On the other hand, if/when some artificial agent has learned the core concepts of causality and language grounding in some small environment - using whatever algorithms are required for that - then it seems plausible that a GPT-like system could be sufficient to provide the mapping to extend these concepts to the whole range of things that we talk about; once the agent understands (according to whatever reasonable nontrivial definition of "understands") the concept of causality for some actions, it can learn the causality of all the other actions which are described if you read all the written text in the world.

peterthehacker 6y ago
This is a great point!
libraryofbabel 6y ago
So goes the old teachers’ proverb: “Tell me and I forget, show me and I may remember, involve me and I understand.”
marmaduke 6y ago
This is a good argument for why AGI is more likely to come from AI gym environments where it learns to control an agent in an environment. It isn’t presented with a static dataset but interacts.

chillacy 6y ago

> cannot separate causality from correlation Neither can most humans without rigorous experimentation. Deep learning today seems to emulate only system 1. It just turns out that much of what expert humans do can be attributed to finely tuned system-1 intuition. To achieve AGI we'd need a system-2. (nomenclature for system-1 and system-2 taken from Kahneman and Tversky's work on human cognition)