3 ms·

I have a different take on it, which I've been saying for years at this point. DeepMind appear to have over-committed to the wrong approach - reinforcement lear

I have a different take on it, which I've been saying for years at this point. DeepMind appear to have over-committed to the wrong approach - reinforcement learning. They've had some great successes with it for sure, but it's always seemed the wrong approach for intelligence, notwithstanding luminaries such as David Silver insisting that "Reward is Enough". The best examples of intelligent systems we have at this point are animal brains, which per cortical architecture(s) appear to operate on an entirely different principle - prediction. The cortex simply put is a prediction machine - not only can it predict, but this is essentially the only thing it does, and ability to predict is the evolutionary selection pressure that created it. It's a bit odd that here in 2023 there is still no widespread agreement in the scientific community how best to define intelligence, but I have my own rather pithy definition that I believe is the most reductive: "Intelligence is the ability to correctly predict (future outcomes) based on prior experience." In other words, intelligence is the capability given to use by our cortex. Why evolution put such a high value on prediction should be pretty obvious : you can now act based on what is going to happen rather than what you immediately perceive. Predict where the food/water is, how the predator/prey are going to behave, etc, etc. Prediction is also the basis of "thought/reasoning" which is essentially just a sequence of (correctly predicted) chained what-if scenarios. It should now be obvious why something as dumb/simplistic as an LLM shows such a "surprising" degree of intelligence - because it is a prediction machine, and why "think step by step" is therefore such a powerful tool. There are probably a dozen or more alternate definitions of intelligence that have been offered by various people, but I believe none are as reductive as mine - other definitions basically describe capabilities that a system/person would have if they were able to predict. Reinforcement learning seems to be a misleadingly attractive idea because subjectively goal-based behavior makes sense to us, but it seems that again this is not reductive enough. The appearance of goal-based behavior is something that comes out of prediction. Part of understanding this is a brilliant insight from the much maligned Jeff Hawkins - prediction (quite literally and mechanistically) becomes action; this occurs because the motor cortex operates just like the rest of the cortex and is a prediction machine - it predicts our motor actions, and those predictive outputs drive our muscles causing the prediction to become true. Goal-based behavior is therefore based on the mechanism of prediction - predict the sequence of what-if actions to achieve a goal, and those actions will be taken. Our self-predictive actions start at birth by predicting (via self-observation) the actions we make at that stage and evolves as life experience causes our predictions to change. TL;DR - DeepMind are riding the wrong horse (RL). Prediction is all you need.

opyate 4y ago
Which of the 3 (reinforcement/supervised/unsupervised) is more amenable to building "expert systems" that can play Go or fold proteins? I saw a thread just earlier [0] where GPT said it invented a game which doesn't yet exist, but it was wrong, unfortunately. Not sure I'd trust it to be an expert in anything. (Can't find it now, but there was also an article recently about Bing/Sydney just getting facts completely wrong.) 0. https://news.ycombinator.com/item?id=35038804 https://news.ycombinator.com/item?id=35038804
88stacks 4y ago
I would argue that intelligence is the ability to survive in your environment. Homeostasis meaning sleeping, eating, drinking, reproducing and avoiding pain. Most of that does require prediction.
- HarHarVeryFunny 4y ago
  > Most of that does require prediction. I think that depends on the type of animal. There are plenty of animals (insects, fish, reptiles) that don't rely on intelligence, but instead survive just by being well adapted to their environment. Behavior in these animals is largely hardwired via genetic coding. Other classes of animals (birds, mammals) have evolved to become more generalists, which required them to evolve intelligence to become more adaptive to diverse environments. So, seeing as evolutionary success isn't inherently tied to intelligence, it seems better not to define it that way. One could still "define" it as that capability that helps provide these generalist classes of animals with some of their survival needs, but that's really only saying what the benefits of intelligence are, not what it actually is. I still think my predictive definition of intelligence is hard to beat, since it seems about as fundamental a definition as is possible.
coyotespike 4y ago
This is a great comment, I've saved it in my notes and I've been thinking about it. Suppose Friston's theory and yours here is basically correct, that prediction is the best measure of and even development of intelligence. Friston thinks (afaict) this controls our motor units too, like Jeff Hawkins whom you cite. GPT uses RL to get better at prediction. How would we do the same for say an agent playing the Atari lunar lander game? We know how to use RL to play Atari. How would we use RL to use prediction to play Atari?

opyate 4y ago

Which of the 3 (reinforcement/supervised/unsupervised) is more amenable to building "expert systems" that can play Go or fold proteins? I saw a thread just earlier [0] where GPT said it invented a game which doesn't yet exist, but it was wrong, unfortunately. Not sure I'd trust it to be an expert in anything. (Can't find it now, but there was also an article recently about Bing/Sydney just getting facts completely wrong.) 0. https://news.ycombinator.com/item?id=35038804 https://news.ycombinator.com/item?id=35038804

88stacks 4y ago

I would argue that intelligence is the ability to survive in your environment. Homeostasis meaning sleeping, eating, drinking, reproducing and avoiding pain. Most of that does require prediction.

HarHarVeryFunny 4y ago
> Most of that does require prediction. I think that depends on the type of animal. There are plenty of animals (insects, fish, reptiles) that don't rely on intelligence, but instead survive just by being well adapted to their environment. Behavior in these animals is largely hardwired via genetic coding. Other classes of animals (birds, mammals) have evolved to become more generalists, which required them to evolve intelligence to become more adaptive to diverse environments. So, seeing as evolutionary success isn't inherently tied to intelligence, it seems better not to define it that way. One could still "define" it as that capability that helps provide these generalist classes of animals with some of their survival needs, but that's really only saying what the benefits of intelligence are, not what it actually is. I still think my predictive definition of intelligence is hard to beat, since it seems about as fundamental a definition as is possible.

coyotespike 4y ago

This is a great comment, I've saved it in my notes and I've been thinking about it. Suppose Friston's theory and yours here is basically correct, that prediction is the best measure of and even development of intelligence. Friston thinks (afaict) this controls our motor units too, like Jeff Hawkins whom you cite. GPT uses RL to get better at prediction. How would we do the same for say an agent playing the Atari lunar lander game? We know how to use RL to play Atari. How would we use RL to use prediction to play Atari?